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PREFACE 


Beginning  in  April  2001,  the  Institute  for  Defense  Analyses  undertook  a  research 
effort  aimed  at  discovering  potential  enhancements  to  the  management  of  the  Missile 
Defense  Agency’s  long-tenn  science  and  technology  (S&T)  program.  The  first  phase  of 
that  effort  produced  a  paper — Science  and  Technology  in  Development  Environments 
(IDA  Paper  P-3764,  May  2003) — that  highlighted  S&T  management  methods  employed 
by  successful  public-  and  private-sector  organizations.  This  paper  follows  up  on  that 
report  by  outlining  how  MDA  could  adapt  and  implement  some  of  these  management 
methods. 

The  authors  wish  to  acknowledge  the  contributions  of  its  distinguished  senior 
advisory  panel,  all  fonner  directors  of  major  S&T  organizations:  Peter  Cannon  (Rockwell 
Science  Center),  Arthur  Chester  (Hughes),  and  Lawrence  Howell  (General  Motors). 
While  their  experiences  shaped  many  of  the  findings  and  recommendations  of  this  report, 
the  final  substantive  and  editorial  judgments  contained  herein,  as  well  as  any  factual 
errors  or  omissions,  are  the  sole  responsibility  of  the  Institute  for  Defense  Analyses. 
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EXECUTIVE  SUMMARY 


Science  and  Technology  in  Development  Environments*  the  precursor  of  this 
Institute  for  Defense  Analyses  (IDA)  study  for  the  Missile  Defense  Agency  (MDA), 
reviewed  science  and  technology  (S&T)  management  methods  employed  by  successful 
public-  and  private-sector  organizations.  This  report  recommends  an  S&T  management 
framework  for  MDA  and  describes  in  detail  four  areas  that  cover  the  range  of  earlier 
work  deemed  applicable  to  MDA’s  mission  and  structure: 

•  Promoting  Radical  Innovation — Fostering  creative  thinking,  developing  new 
missile  defense  concepts,  and  supporting  emerging  technologies. 

•  Strategic  Technology  Steering  and  Execution — Managing  a  portfolio  of  S&T 
projects  in  a  limited  number  of  technology  areas  critical  to  the  long-term  MDA 
mission. 

•  Networking  and  Outreach — Interacting  with  external  communities  to  identify 
promising  technologies,  emerging  threats,  gaps  in  research,  and  partnering 
opportunities,  as  well  as  sustaining  internal  support  for  the  MDA  research 
program. 

•  Analytical  Techniques  and  Modeling  and  Simulation — Supporting  rigorous 
S&T  planning  and  management  within  MDA  using  modern  analytic 
techniques,  tools  and  methods. 

IDA  also  undertook  a  “pilot  technology  assessment”  that  experimented  with  some 
of  these  methods  to  identify  specific  candidate  long-term  S&T  investments,  looking  out 
to  the  2015-2020  time  frame.  IDA  staff  (and  an  independent,  parallel  effort  at  Sandia 
National  Laboratories)  developed  innovative  missile  defense  concepts  and  identified  and 
assessed  technologies  that  could  enable  them.  Pilot  project  teams  employed  a  systematic, 
top-down  analysis  process  that  proceeded  from  high-level  missile  defense  challenges 
(boost-phase  and  tenninal-phase  missile  defeat),  to  a  functional  description  of  novel 
operational  concepts  for  addressing  these  challenges,  to  high-level  systems  descriptions 
that  might  realize  the  functions,  to  technical  capabilities  that  could  enable  such  systems 


R.  Van  Atta,  R.  Bovey,  J.  Harwood,  W.  Hong,  A.  Hull,  B.  Kindberg,  and  M.  Lippitz,  Science  and 
Technology’  in  Development  Environments,  IDA  Paper  P-3764  (Alexandria,  Va.:  Institute  for  Defense 
Analyses,  May  2003). 
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to  be  built.  A  companion  study  describes  the  concepts  developed  and  S&T  investment 
candidates  identified./ 


OVERALL  ORGANIZATION  AND  MANAGEMENT  OF  MDA  STRATEGIC  S&T 


Figure  ES-1  summarizes  a  proposed  organizational  framework  for  implementing 
the  S&T  management  methods  recommended  in  this  report.  MDA/AS  refers  to  the 
Director  for  Advanced  Systems  within  the  current  MDA  organization.  This  individual 
would  provide  executive  focus  akin  to  the  role  often  played  by  a  Chief  Technology 
Officer  in  a  major  corporation.  Two  new  functions  would  be  added  to  current  MDA/AS 
responsibilities:  strategic  technology  steering  and  innovation  promotion  activities  under  a 
Chief  Innovation  Executive.  Each  of  these  functions  can  be  built  gradually,  beginning 
with  a  low  level  of  staffing. 


MDA/AS  (“CTO”) 


Support 

•  Networking 

•  Analytics 

•  Organizational 


Strategic  Current  AS 
Technology  activities 

Steering 


Chief  Innovation 
Executive  (CIE) 


Strategic  Program- focused  Emerging  Advanced  Advanced 

Technologies  S&T  Technology*-:*  Concept  Systems 

^'-'^Development  Experimentation 


Figure  ES-1.  Recommended  MDA/AS  Long-Term  S&T  and  Innovation  Operation 

The  Chief  Innovation  Executive  would  lead  efforts  aimed  at  fostering 
technological  and  conceptual  innovation  to  address  broad  missile  defense  challenges.  An 
emerging  technology  group  would  scout,  monitor,  assess,  and  support  immature 
technologies  that,  if  successfully  developed,  could  facilitate  breakthrough  improvements 
in  ballistic  missile  defense  systems,  subsystems,  and  components.  These  groups  would 
work  closely  and  iteratively  with  advanced  concept  development  teams  to  regularly 
propose,  revise,  and  revisit  concepts  based  on  changing  technologies  and  challenges. 
(The  top-down  concept  development  and  technology  identification  effort  explored  in  the 


t  Richard  Van  Atta  et  al.,  Results  of  the  Technology  Assessment  Pilot  Project  for  the  Missile  Defense 
Agency  (Alexandria,  Va.:  Institute  for  Defense  Analyses,  forthcoming). 
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pilot  technology  assessment  is  one  method  that  would  be  used.)  The  advanced  systems 
experimentation  group  would  undertake  proof-of-concept  projects  for  a  small  number  of 
concepts  matured  through  the  interaction  of  emerging  technology  and  advanced  concept 
development  efforts,  with  the  goal  of  determining  whether  the  integrated  capabilities 
entailed  should  be  considered  as  a  candidate  for  development. 

Strategic  technology  steering  would  be  a  management  structure  and  set  of 
activities  for  directing  a  portfolio  of  long-term  S&T  projects  in  specific  technology  areas 
deemed  to  be  critical  to  the  long-term  success  of  the  MDA  mission.  Support  personnel 
could  be  brought  on  to  assist  the  MDA/AS  organization  through  networking,  outreach, 
analysis,  and  organizational  process  management.  Over  time,  MDA  could  consider 
forming  high-level  executive  groups  to  help  establish  a  long-range  vision,  balance  the 
overall  S&T  program  with  respect  to  MDA  strategic  objectives,  and  overcome 
organizational  barriers  to  the  pursuit  of  new  concepts. 

PROMOTING  RADICAL  INNOVATION 

MDA’s  mission  objective  raises  complex  capability  challenges  requiring 
significant  innovation  over  the  next  few  decades.  Effectively  encouraging  radical 
innovation  requires  modification  of  traditional  management  processes  and  technology 
competencies.  Innovation  cannot  be  produced  on  command  or  on  a  fixed  schedule,  but 
well-designed  processes  can  enable  the  right  people  to  be  vastly  more  effective  and 
create  a  culture  of  innovation.  Structured  and  informal  interactions  among  people  focused 
on  technology  and  people  focused  on  applications  (both  internal  and  external  to  the 
organization),  along  with  linkages  to  overall  organizational  strategy,  are  critical  factors 
that  drive  the  evolution  of  an  innovative  organization.  The  following  specific  actions 
could  be  implemented  over  time: 

•  Appoint  a  Chief  Innovation  Executive  to  lead  and  support  the  overall  effort. 

•  Recruit  concept  managers — These  are  individuals  respected  for  their  technical 
acumen  who  can  bring  together  technologists,  systems  integrators,  and 
operational  experts  from  a  variety  of  stakeholder  organizations  to  identify  and 
articulate  the  promise  of  a  concept  or  technology,  encourage  its  development, 
and  gamer  top-level  support. 

•  Form  cross-functional  innovation  teams  to  systematically  pursue  mission  needs 
discovery  and  definition,  provide  continuous  technology  monitoring  and 
assessment,  build  and  maintain  a  network  of  technical  and  application 
expertise  relevant  to  MDA  strategic  challenges,  perform  regular  outreach  in 
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important  technology  communities,  and  provide  professional  support  for 
project  and  process  management. 

•  Support  external  S&T  community  engagement  to  remain  cognizant  of 
emerging  technologies  and  foster  new  concepts. 

•  Develop  management  structures  and  internal  capabilities,  such  as  a  customized 
“technology  stage  gate”  process  for  screening  and  managing  S&T  candidate 
projects,  independent  experts  groups,  high-level  review  boards,  and  a 
knowledge  base  of  past  and  present  MDA  S&T  programs  and  innovation 
concepts. 

•  Form  a  separate  Advanced  Systems  Experimentation  Office  focused  on  proof- 
of-concept  experimentation  and  system-of-systems  demonstration. 

STRATEGIC  TECHNOLOGY  STEERING  AND  EXECUTION 

Strategic  technology  steering  comprises  a  management  structure  and  set  of 
activities  useful  for  directing  a  portfolio  of  long-term  S&T  projects  in  a  specific 
technology  area  determined  to  be  critical  to  the  long-term  success  of  the  MDA  mission. 
An  effective  technology  steering  activity  provides  consistent  long-tenn  engagement  and 
support  for  critical  technology  needs  to  maximize  S&T  productivity,  effectiveness,  and 
timeliness.  Mechanisms  can  include  identification  of  S&T  gaps  and  opportunities  and 
subsequent  resource  allocation,  facilitation  of  internal  and  external  technical 
communications  and  collaborations,  formal  technology  management  processes,  and 
development  of  technical  personnel.  Technology  steering  should  be  sufficiently  flexible 
to  weather  organizational  restructurings  and  changing  management  personalities, 
organizational  priorities,  and  mission  needs.  Specific  steps  over  time  could  include  the 
following: 

•  Establishing  a  strategic  technology  steering  group  focused  on  a  selected 
technology,  with  a  respected  leader  and — critically — direct  reporting  to 
supportive,  committed  top  management. 

•  Supporting  the  strategic  technology  steering  group  with  networking  of  related 
technology  communities,  potentially  including  creation  of  industry  panels  to 
allow  contractor  participation. 

•  Implementing  a  technology  stage-gate  process  customized  to  address  MDA’s 
mission,  planning  horizon,  and  contract  environment. 

•  Developing  virtual  laboratories:  a  highly  collaborative  fonn  of  S&T 
outsourcing  employing  formal,  precompetitive  coalition  contracting  structures 
to  permit  interaction  of  companies  on  S&T  while  protecting  intellectual 
property  rights. 
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NETWORKING  AND  OUTREACH 


“Networking”  refers  to  the  processes  used  to  interact  with  science  and 
engineering  communities  inside  and  outside  MDA  to  identify  promising  technologies, 
potential  emerging  threats,  gaps  in  current  research,  and  partnering  opportunities. 
Outreach,  typically  directed  toward  MDA  executive  and  program  leaders,  aims  to  gain 
support  for  a  healthy  long-term  research  program  by  demonstrating  the  need  for  it  and 
documenting  how  it  is  supporting  the  MDA  mission. 

ANALYTICAL  TECHNIQUES  AND  MODELING  &  SIMULATION 

The  planning  and  management  of  MDA’s  S&T  program  involves  a  wide  range  of 
functions,  among  them  environmental  scanning,  development  of  research  strategies  to 
guide  program  planners  and  managers,  determination  of  S&T  focus  areas,  identification 
and  development  of  project  concepts,  selection  of  research  projects,  management  of 
ongoing  projects,  transition  of  research  results,  and  infrastructure  capabilities  to  support 
the  research  program.  A  large  number  of  tools  are  available  to  support  these  S&T 
planning  and  management  functions.  Two  examples  of  techniques  MDA/AS  should 
consider  for  near-tenn  implementation  are  early  stage-gate  processes  and  scoring  models 
for  project  evaluation  and  selection. 

NEXT  STEPS 

The  S&T  management  structures  outlined  here  are  general  functions  that  have 
taken  into  account  the  general  MDA  mission  and  working  environment  but  have  not  been 
specifically  defined.  To  translate  these  ideas  into  formal  processes  for  seeking, 
identifying,  maturing,  and  eventually  deciding  upon  options,  MDA  personnel  will  need  to 
be  directly  involved  in  implementation,  with  MDA/AS  actively  engaged  and  overseeing 
the  process.  As  next  steps  in  MDA  implementation,  IDA  proposes  the  following: 

•  Prepare  an  MDA/AS  Strategic  S&T  Implementation  Plan,  including 
delineating  the  interrelationships  among  the  proposed  MDA(AS)  S&T 
activities  and  other  aspects  of  MDA  S&T  management,  such  as  systems 
engineering,  mission  analysis,  and  systems  architecture  planning. 

•  Conduct  workshops  for  MDA  leadership  and  key  participants  to  foster  a 
uniform  understanding  of  the  S&T  strategic  concept  and  to  develop  the  needed 
customized  processes  and  tools. 

•  Complete  current  technology  assessment  studies  and  add  new  ones  (e.g., 
midcourse  defeat  or  close-in  missiles).  This  should  include  doing  the 
following: 
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o  Undertaking  detailed  engineering  assessments  for  selected  concepts, 
leading  to  identification  of  specific  technology  gaps, 
o  Making  risk/cost  assessment  of  identified  technologies  and  consolidating 
these  analyses  into  composite  “grand  challenges”  across  mission  areas  as 
well  as  candidate  strategic  technologies, 
o  Performing  validation  experiments  on  promising  concepts  (e.g., 
explosively  formed  projectiles). 

•  Appoint/designate  a  Chief  Innovation  Executive  and  start  building  an 
innovation  promotion  organization  to  pursue  identified  key  technology 
challenges,  better  define  and  document  mission  needs,  systematically  search 
for  emerging  technologies,  and  develop  additional  new  concepts. 

•  Undertake  formal  networking  activities  to  establish  linkages  with  other 
government  programs  and  to  identify  potential  partners  for  future  strategic 
technologies. 

•  Begin  building  an  S&T  candidate  projects  database  to  serve  as  an 
“organizational  memory,”  enabling  periodic  revisiting  of  concepts  and  tracking 
of  emerging  technologies. 

•  Establish  a  traceable,  quantitative  scoring  system,  to  permit  concepts  and 
technologies  to  be  consistently  ranked  across  concepts  and  over  time. 

•  Assess  options  for  virtual  laboratories  for  ongoing  long-term  S&T  execution. 

•  Determine  the  appropriate  amount  and  type  of  modeling  and  simulation  to 
support  MDA  S&T  planning. 
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I.  OVERALL  ORGANIZATION  AND  MANAGEMENT  OF 

MDA  STRATEGIC  S&T 


Richard  Van  Atta,  Michael  Lippitz,  and  Robert  Bovey 

The  precursor  of  this  Institute  for  Defense  Analyses  (IDA)  study  for  the  Missile 
Defense  Agency  (MDA)1  examined  the  science  and  technology  (S&T)  management 
methods  of  public-  and  private-sector  organizations  that  had  proved  successful  over  time 
in  building  and  maintaining  dominant  capabilities.  The  chapters  that  follow  suggest  how 
to  begin  implementing  within  MDA  some  of  the  most  important  categories  of  these 
methods:  promoting  radical  innovation  (Chapter  II);  strategic  technology  steering  and 
execution  (Chapter  III);  networking  and  outreach  (Chapter  IV);  analytical  techniques 
(Chapter  V);  and  modeling  and  simulation  (Chapter  VI).  This  introductory  chapter 
discusses  the  underlying  ideas  and  context  of  these  S&T  management  methods  and 
outlines  how  they  might  be  divided  functionally  within  the  MDA  Advanced  Systems  and 
merged  with  existing  MDA  management  processes. 

UNDERLYING  IDEAS  AND  CONTEXT 

MDA  was  formed  in  January  2002  to  consolidate  Department  of  Defense  (DoD) 
ballistic  missile  defense  system  (BMDS)  programs.  Service  Operational  Requirements 
Documents,  which  strictly  defined  desired  system  specifications  for  programs  brought 
into  MDA,  were  cancelled.  Instead,  an  evolutionary,  capability-based  development 
approach  was  adopted.  Under  this  approach,  the  overall  BMDS  was  to  evolve  toward  its 
ultimate  objective — an  integrated  BMDS  capable  of  providing  a  layered  defense  for  the 
homeland,  deployed  forces,  friends,  and  allies  against  ballistic  missiles  of  all  ranges  in  all 
phases  of  flight2 — by  deploying  an  initial  system  based  on  existing  technologies,  followed 
by  a  series  of  follow-on  “blocks”  incorporating  improved  technologies  as  they  become 
available.  This  approach,  known  as  “spiral  development,”  stands  in  contrast  to  the 
traditional  requirements-based  development  method  (represented  in  the  previous 


1  R.  Van  Atta,  R.  Bovey,  J.  Harwood,  W.  Hong,  A.  Hull,  B.  Kindberg,  and  M.  Lippitz,  Science  and 
Technology  in  Development  Environments,  IDA  Paper  P-3764  (Alexandria,  Va.:  Institute  for  Defense 
Analyses,  May  2003). 

2  MDA  mission  statement,  http://www.acq.osd.mil/bmdo/bmdolink/html/mission.html,  1  March  2004. 
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Operational  Requirements  Documents),  in  which  the  ultimate  capability  and  technical 
approach  were  defined  in  advance,  based  on  a  well-specified  threat. 

In  a  capabilities-based,  spiral-development  approach,  S&T  research  aims  to 
identify  and  assess  any  technology  advances  that  can  meaningfully  increase  capabilities 
beyond  what  can  be  achieved  currently.  Internal  guidance  documents  call  for  MDA  to 
“examine  the  widest  possible  range  of  options  to  increase  system  capabilities... 
include(ing)  assessing  innovative  approaches  and  new  technology  concept  employment 
for  land,  air,  sea  and  space-based  application,  potentially  inserting  enhancements  at  all 
levels  of  the  BMDS:  component,  element,  and  system.”3  In  the  course  of  our  research,  we 
identified  several  ways  to  develop  candidates  for  research  work; 

•  Top-down  mission  analysis  (such  as  the  pilot  technology  assessment). 

•  Project  reviews  that  identify  shared  problems  or  solutions. 

•  External  and  internal  solicitations  for  new  ideas,  supported  with  appropriate 
funding. 

•  Technology  scouting  and  assessment  for  applicability  to  the  mission. 

•  Creative  idea  generation  methods  (such  as  providing  staff  with  seed  money  to 
conceive  and  articulate  novel  ideas). 

Several  different  types  of  research  can  emerge  from  these  types  of  processes: 

•  Strategic  technologies — technologies  recognized  as  having  long-term  potential 
impact  across  several  potential  BMDS  components. 

•  Grand  challenges — critical  general  needs,  without  reference  to  particular 
technological  means. 

•  Solution  paths — specific  approaches  to  solving  a  defined  BMDS  problem. 

To  develop  significant  new  capabilities  for  future  blocks,  MDA  will  need  to 
combine  various  research  approaches  into  a  robust,  systematic,  and  strategically  focused 
S&T  effort.  Because  needed  capabilities  will  not  depend  on  a  particular  technical 
approach  (which  might  be  infeasible)  or  a  particular  definition  of  the  threat  (which  might 
change),  the  overall  effort  should  blend  top-down,  leadership-driven  efforts  with  bottom- 
up,  opportunity-driven,  entrepreneurial  processes.  It  should  blend  the  following: 

•  Long-term  focus  on  strategic  technologies  in  well-defined  domains  with  quick- 
turn  exploration  of  crosscutting  new  concepts. 


3  BMDS  Capability  Enhancement  Process,  Version  5.0,  5  February  2003,  Section  1.2,  p.  5. 
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•  Open-ended  scientific  experimentation  to  characterize  unknown 
phenomenology  with  focused  systems  engineering  to  prove  out  integrated 
capabilities. 

•  Reliance  on  internal  scientific  and  application  expertise  with  leveraging  of 
external  sources  of  innovation.  Where  appropriate,  it  should  apply  rigorous 
systems  analysis  and  metrics,  but  leaving  room  for  open-ended  (but  not 
indefinite)  exploration  in  other  areas. 

An  example  of  this  type  of  strategically  focused  S&T  management  occurred  in  the 
1970s,  when  U.S.  and  North  Atlantic  Treaty  Organization  (NATO)  planners  and  policy 
makers  supported  sustained  concept  development  efforts  to  better  define  the  Soviet 
military  challenge  and  develop  alternative  responses  to  nuclear  war.  Panels,  boards,  and 
conferences,  attended  by  both  government  personnel  and  independent  experts,  played  an 
important  role  in  the  development,  communication,  review,  and  refinement  of  concepts. 
Although  these  gatherings  were  sponsored  by  government  organizations,  organizational 
agendas  and  detailed  mission  requirements  did  not  constrain  consideration  of 
controversial  ideas,  and  intermediate  government  organizations  were  often  bypassed. 
Over  time,  these  deliberations  converged  on  defense  concepts  that  emphasized  standoff 
precision  strike.  The  challenge  of  standoff  precision  strike  was  defined  in  detail, 
recognizing  the  need  for 

integration  of  a  wide  range  of  technologies:  target  detection,  recognition 
and  location;  delivery  vehicles  and  munitions;  and  weapon  navigation  and 
guidance.  This  dictate(d)  a  unified  approach  to  development  in  these  areas 
and  the  establishment  of  operational  procedures  for  effective  integration 
and  employment  of  both  targeting  and  weapons  systems.4 

Respected  defense  analysts  (involved  in  the  deliberations)  promoted  standoff 
precision-strike  concepts  throughout  the  defense  community  and  to  top  Office  of  the 
Secretary  of  Defense  (OSD)  and  Service  leadership.  With  the  eventual  imprimatur  of  top 
DoD  leadership,  resources  increasingly  focused  on  supporting  strategic  technologies  and 
systems  demonstrations  associated  with  various  precision-strike  capabilities.  These  efforts 
are  credited  with  fostering  the  conventional  military  superiority  that  the  United  States 
enjoys  today.5 


4  “Final  Report  of  the  Advanced  Technology  Panel,”  ARPA/DNA  Long  Range  R&D  Planning ,  April  30, 
1975,  p.  6. 

5  Van  Atta  et  al.  Science  and  Technology,  pp.  10-11. 
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To  test  out  the  promise  of  such  approaches  in  the  current  MDA  context,  IDA 
undertook  a  pilot  project  aimed  at  developing  novel  missile  defense  concepts  and 
identifying  needed  technologies.  Small  groups  of  technical  and  applications  experts 
focused  on  a  few  particular  BMDS  challenges.  Through  brainstorming,  they  identified  a 
number  of  operational  concepts  that  could  potentially  meet  the  challenges.  A  few  of  the 
more  promising  operational  concepts  were  selected  for  more  detailed  examination, 
leading  to  high-level  description  of  hypothetical  systems  that  could  carry  out  the 
operational  concept.  These  systems  concepts  were  reviewed  to  identify  technology 
capabilities  that  might  enable  them.  These  technologies  were  assessed,  and  in  some  cases 
relatively  broad  areas  of  S&T  in  which  research  might  well  produce  the  knowledge 
needed  to  enable  one  or  more  of  these  technical  capabilities  were  identified.  Finally,  the 
group  proposed  possible  components  of  a  specific,  defined  S&T  program.  Throughout  the 
process,  the  group  assessed  and  set  priorities  among  concepts,  systems,  technology 
capabilities  and  S&T  areas.  (The  detailed  findings  and  conclusions  of  these  efforts  are 
described  in  Volume  2.) 

The  Buried  Autonomous  Anti-Missile  System  (BAAM)  is  one  concept  that 
illustrates  how  supporting  synthetic  thinking  and  systematic  analyses  can  generate  new 
solutions  to  important  BMDS  problems.  BAAM  contemplates  deployment  of  small 
interceptor  systems  near  enemy  missile  launch  sites.  These  interceptor  systems  would 
detect  local  launches  and  attack  the  enemy  missiles  during  their  boost  phase.  If 
successfully  developed,  such  systems  could  offer  policy-makers  important  new  options  in 
addressing  ballistic  missile  threats.  Whereas  most  existing  BMDS  concepts  contemplate  a 
large-scale,  integrated  capability  in  which  individual  pieces  are  only  effective  in  the 
context  of  a  complex,  layered  system  of  systems,  BAAM  could  permit  numerous  small, 
independent  systems  to  be  deployed.  It  would  provide  policy-makers  with  important  new 
political  and  military  options,  transforming  an  important  component  of  the  ballistic 
missile  defense  problem.  Many  questions — technical,  political,  practical — need  to  be 
answered  before  pursuing  such  a  system. 

AN  ORGANIZATIONAL  REALIZATION 

The  top-down  concept  development  and  technology  assessment  effort,  undertaken 
as  a  pilot  project,  is  just  one  example  of  the  type  of  productive  interaction  that  can  be 
fostered  between  applications-oriented  and  technology-oriented  people.  Figure  1-1 
summarizes  how  various  scientific,  engineering,  and  management  elements  of  a  long¬ 
term  MDA  S&T  program  could  be  realized  organizationally,  by  balancing  and  channeling 
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top-down  and  bottom-up  forces.  MDA/AS  refers  to  the  Director  for  Advanced  Systems 
within  the  current  MDA  organization.  MDA/AS  would  provide  executive  focus  for  the 
overall  MDA  S&T  effort,  akin  to  the  role  often  played  by  a  Chief  Technology  Officer  in  a 
major  corporation.  “Current  AS  Activities,”  covers  the  current  functions  of  the  MDA/AS 
office,  which  were  not  investigated  as  part  of  the  IDA  project.  The  other  elements  in  the 
figure  do  not  currently  exist  in  any  structured,  authoritative  way  within  the  MDA/AS 
office,  and  would  have  to  be  added.  The  new  functions  proposed  here  cover  the  missing 
S&T  areas  needed  to  support  MDA:  Innovation,  the  systematic  exploration  of  new 
concepts  and  technology  applications,  and  strategic  technology,  the  systematic  advance  of 
technologies  known  to  be  critical  to  MDA’s  mission,  regardless  of  specific  system 
implementation. 


Support 


MDA/AS  f“CTO”I -  -  •  Networking 


Strategic  Current  AS 
Technology  activities 

Steering 


Chief  Innovation 
Executive  (CIE) 


Strategic  Program- focused  Emerging  Advanced  Advanced 

Technologies  S&T  Technology*--;*  Concept  Systems 

*;-*Developmcnt  Experimentation 


Figure  1-1.  Recommended  MDA  (AS)  Long-Term  S&T  and  Innovation  Operation 

The  Chief  Innovation  Executive  would  lead  a  function  that  would  foster  and  seek 
to  build  upon  technological  and  conceptual  innovation  in  meeting  missile  defense  mission 
challenges.  This  position,  which  reports  directly  to  the  director  of  MDA/AS,  would  be 
designed  to  provide  management  stability  through  changes  in  MDA  leadership.  Long¬ 
term  stability  is  needed  because  some  of  the  projects  and  processes  here  may  need 
continuity  and  consistent  direction  for  a  number  of  years  to  reach  fruition.  The  Chief 
Innovation  Executive  would  be  primarily  responsible  for  overseeing  the  development  of 
the  research  agenda;  providing  the  justification  needed  to  support  that  agenda;  overseeing 
the  execution  of  the  program:  and  recommending  increases,  reductions,  or  elimination  of 
funding  for  individual  research  projects. 
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Reporting  to  the  Chief  Innovation  Executive  would  be  groups  focused  on 
emerging  technology,  advanced  concepts  development,  and  advanced  systems 
experimentation.  Emerging  technology  and  advanced  concept  development  form  the  core 
of  the  innovation  operation.  Emerging  technologies  responsibility  would  encompass 
scouting,  monitoring,  assessing,  and  supporting  emerging  technologies  that  could 
facilitate  breakthrough  improvements  in  BMDS,  subsystems,  and  components.  The  top- 
down  concept-development  and  technology-identification  effort  explored  in  the  pilot  tech 
assessment — moving  explicitly  from  challenges  to  concepts  to  systems  to  technologies  to 
S&T  candidates — is  one  method  that  would  be  used  as  part  of  an  ongoing  advanced 
concept  development  effort  that  would  regularly  propose,  revise,  and  revisit  concepts 
based  on  changing  technologies  and  challenges.  Advanced  systems  experimentation 
would  undertake  proof-of-concept  projects  for  those  concepts  that  have  matured 
sufficiently  through  the  interaction  of  emerging  technology  and  advanced  concept 
development  efforts  and  that  are  deemed  to  have  the  greatest  promise  for  addressing 
BMDS  challenges.  The  goal  of  advanced  systems  experimentation  would  be  to  determine 
whether  the  integrated  capabilities  entailed  should  be  considered  as  candidates  for  formal 
development.  As  discussed  in  Chapter  II,  these  groups  could  start  with  only  a  few  people 
and  evolve  gradually. 

Strategic  technology  steering  would  be  another  branch  of  the  proposed  new 
MDA/AS  innovation  organization.  This  organization  would  oversee  a  variety  of  activities 
for  overseeing  and  directing  a  portfolio  of  long-term  S&T  projects  in  a  small  number  of 
technology  areas  deemed  to  be  critical  to  the  long-term  success  of  the  MDA  mission. 
Over  time,  MDA  could  consider  forming  high-level  executive  groups  to  support 
MDA/AS  and  the  Chief  Innovation  Executive  in  establishing  a  long-range  vision  and 
support  for  long-term  research,  focusing  and  balancing  the  overall  S&T  program  with 
respect  to  MDA  strategic  objectives,  and  to  assist  in  overcoming  organizational  barriers  to 
the  pursuit  of  new  concepts.  Also  over  time,  support  personnel  could  be  brought  on  to 
assist  all  of  these  MDA/AS  efforts  in  networking,  outreach,  analysis,  and  organizational 
process  management. 
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INTEGRATION  WITH  TOP-LEVEL  MANAGEMENT  SYSTEMS 


" Innovative  and  scientific  excellence  are  not  sufficient — an  organization  must 
have  mechanisms  to  link  innovation  to  the  businesses  and  customers,  as  well  as  to 
transition  new  developments  from  R&D  to  operations  and  the  marketplace.  ” 6 

MDA’s  central  research  program  will  require  focus  and  management  expertise  to 
produce  results.  As  a  practical  matter,  it  will  be  executed  primarily  by  government 
contractors,  but  it  must  be  managed  with  sufficient  independence  and  crosscutting 
expertise  to  facilitate  innovative  approaches.7  MDA’s  program  elements  are  similar  to 
many  industry  organizations  in  which  semi-autonomous  business  units  develop  distinct 
systems.  In  such  settings,  central  research  in  MDA  needs  to  provide  support  that  includes 
the  following: 

•  Seeking  and  helping  solve  critical  problems  for  program  elements,  in  an 
internal  consulting  role  (hence  building  organizational  support  for  central 
research). 

•  Identifying  new  technologies  that  can  either  be  integrated  into  existing  systems 
or  spawn  new  systems. 

•  Covering  the  technical  gaps  between  program  elements. 

•  Identifying  common  problems  and  solutions  across  program  elements. 

•  Supporting  technologies  that  span  more  than  one  program  element,  especially 
potential  strategic  technologies.8 

•  Seeking  revolutionary  new  approaches  to  BMDS. 

•  Acting  as  networking  center  to  bring  ideas  into  MDA  and  to  circulate  ideas 
across  MDA  program  elements. 

To  strengthen  long-term  research  and  innovation  programs  at  MDA  in  this  way,  a 
concerted  effort  by  the  leadership  of  MDA  over  an  extended  period  will  be  required.  This 
effort  will  include  the  following: 


6  Van  Atta,  et  at.,  Science  and  Technology. 

7  Chapter  III  describes  a  variety  of  outsourcing  structures  that  can  create  a  robust  research  organization 
across  technology  areas  beyond  the  scope  of  a  single  organization.  Other  options  available  for  MDA 
include  increases  in  the  MDA/AS  workforce  or  support  contracting,  support  from  a  FFRDC,  and 
outsourcing  a  defined  scope  of  work  to  a  single  S&T  organization  such  as  the  Naval  Research 
Laboratory  or  the  Army  Research  Laboratory. 

8  The  MDA  Enhancement  Plan  is  a  vehicle  for  coordinating  technology  needs,  development  plans,  costs, 
and  schedules  across  MDA.  It  (1)  provides  the  mix  of  concepts  and  technologies  available  to  augment 
expected  Block  capabilities;  (2)  maps  these  concepts  and  technologies  into  augmentation  roles;  and  (3) 
details  processes  for  maturing,  assessing,  and  infusing  concepts  and  technology. 
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•  Increasing  and  stabilizing  funding  for  long-term  research  and  fostering 
innovation. 

•  Making  the  organizational  changes  needed  to  establish  and  maintain  the 
relevance  of  long-term  research. 

•  Visibly  promoting  the  need,  both  within  and  outside  MDA,  for  focused  long¬ 
term  research,  to  support  an  effective  S&T  program  in  an  organization  that  is 
currently  focused  on  implementation. 

•  Supporting  MDA/AS  in  implementing  the  long-tenn  research  program  and  in 
proving  innovative  missile  defense  concepts. 

The  S&T  management  structure  established  in  MDA  must  maintain  a  balance 
between  necessary  program  reviews  from  upper  management  and  freedom  to  explore 
innovative  technologies  and  solution  paths  that  tend  to  move  forward  in  fits  and  starts, 
with  many  dead  ends  along  the  way.  There  must  also  be  a  balance  struck  between  the 
need  to  provide  funding  for  a  long-term  research  agenda  and  the  need  to  support  the 
immediate  research  challenges  of  current  programs.  The  percentages  devoted  to  each  type 
of  research  will  change  over  time,  depending  on  immediate  needs  and  the  strategic 
direction  of  the  organization.  However,  if  top  management  fails  to  protect  some 
percentage  of  the  central  research  budget  for  long-term  research  and  radical  innovation, 
those  funds  will  tend  to  be  consumed  by  immediate  needs.  The  need  for  patience  and  a 
long-term  vision  from  top  management  was  one  of  the  most  consistent  themes  in  the 
precursor  IDA  report  on  S&T  Management. 

Over  time,  MDA  could  consider  forming  executive  boards  to  support  MDA/AS 
and  the  Chief  Innovation  Executive  in  establishing  a  long-range  vision  and  support  for 
long-term  research,  focusing  and  balancing  the  overall  S&T  program,  and  overcoming 
organizational  barriers  to  the  pursuit  of  new  concepts.  Such  boards  could  provide  stability 
through  changes  in  leadership.  Actual  membership,  charter,  and  operating  procedures  will 
have  to  be  developed  with  MDA  but  initial  thoughts  are  provided  below: 

•  Corporate  S&T  Oversight  Board — Members  would  include  MDA’s  Deputy 
Director,  Technical  Director,  and  Technical  Advisor.  The  Board  would  meet 
approximately  twice  a  year:  once  as  part  of  the  budget  process  and  a  second 
time  for  a  program  review.  Its  primary  functions  should  be  to  evaluate  the  long¬ 
term  research  program  with  respect  to  MDA’s  strategic  objectives,  provide 
final  approval  of  the  research  agenda,  and  defend  that  agenda  throughout 
MDA. 

•  S&T  Management  Board — This  board  would  be  chaired  by  the  director 
MDA/AS  and  contain  senior  members  from  each  of  the  individual  MDA 
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system  development  programs.  The  primary  functions  of  the  board  would  be 

(1)  to  serve  as  a  forum  for  the  system  development  programs  to  recognize 
common  long-term  S&T  issues,  which  might  then  be  funded  by  MDA/AS,  and 

(2)  to  offer  perspectives  on  new  concepts  being  pursued  by  MDA/AS  and  how 
the  underlying  capabilities  might  transition  into  future  missile  defense  systems. 
The  result  of  these  deliberations  would  be  a  portfolio  approach  to  the  MDA 
S&T  research  agenda  that  supports  the  strategic  objectives  of  MDA  (beyond 
just  the  incremental  needs  of  current  systems  programs),  consistent  with  the 
funding  goals  and  the  direction  provided  by  the  Corporate  S&T  Oversight 
Board. 

In  addition  to  top-level  management  boards,  the  new  concepts  outlined  in  this 
report  will  need  to  be  integrated  with  S&T  management  systems  that  MDA  is  in  the 
process  of  establishing,  as  described  in  the  “BMDS  Integrated  Program  Plan.”  Figure  1-2 
summarizes  this  management  system.  Many  of  the  ideas  described  in  the  following 
chapters  can  be  fit  to  this  structure.  For  example,  the  AS  database  could  serve  as  the 
“organizational  memory”  called  for  in  Chapters  II  and  III.  Networking  functions 
described  in  Chapter  IV  are  consistent  with  the  BMDS  Enhancement  Plan,  which  “will 
capture  necessary  information  on  external  technology  programs  relevant  to  MDA  (e.g., 
DARPA  [Defense  Advanced  Research  Projects  Agency]  projects,  commercial 
technologies).”  The  “BMDS  Utility  Analysis”  could  be  implemented  as  a  Technology 
Value  Model,  one  of  the  analytic  efforts  described  in  Chapter  VI  aimed  at  fostering 
consistently  applied,  quantitative  judgment  year  after  year  so  that  comparisons  can  be 
made  between  judgments  made  in  different  years  and  mission  sub-areas.  Beyond  the 
items  in  Figure  1-2,  strategic  challenges  would  be  informed  by  the  MDA  Technical 
Objectives  and  Goals  documents,  a  view  of  threat  in  the  Adversary  Capability  Document, 
and  experiments  run  using  the  Adversary  Vignettes  Database.  The  options  presented  to 
the  Director  MDA  for  decision  will  be  worked  out  between  MDA/AS,  the  MDA 
Operational  Concept  Team,  the  Joint  National  Integration  Center,  and  others.  Annexes  to 
the  management  documents  above  may  be  required  to  guide  teams  working  to  build  a 
strategically  focused  S&T  program. 
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Figure  1-2  MDA’s  Technology  Development  Process 
(Source:  BMDS  Integrated  Program  Plan,  11  June  2003  draft,  Figure  22) 


MDA/AS  should  prepare  a  Strategic  S&T  Implementation  Plan  to  delineate  the 
interrelationships  among  the  proposed  MDA/AS  future-oriented  S&T  activities  and  other 
aspects  of  MDA  R&D  management,  such  as  systems  engineering,  mission  analysis,  and 
systems  architecture  planning.  That  plan  would  need  to  include  a  clear  and  agreed 
statement  of  the  MDA/AS  mission  because  it  concerns  long-term  S&T  and  innovation, 
beyond  currently  planned  BMDS  systems.  The  plan  should  include  resource 
commitments  and  authorities,  and  it  should  articulate  the  types  of  S&T  strategies  that  will 
be  pursued  and  key  processes  to  be  employed.  The  chapters  that  follow  describe  several 
types  of  recommended  strategies  and  processes  that  MDA  can  adapt  to  its  particular 
needs. 
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II.  PROMOTING  RADICAL  INNOVATION  IN  MDA 


Michael  J.  Lippitz  and  Robert  C.  Wolcott 


OVERVIEW 

MDA’s  mission  objective  is  to  “develop  and  field  an  integrated  BMDS  capable  of 
providing  a  layered  defense  for  the  homeland,  deployed  forces,  friends,  and  allies  against 
ballistic  missiles  of  all  ranges  in  all  phases  of  flight.”1  This  mission  raises  broad  and 
complex  challenges.  In  addition  to  standard  engineering  and  technology  development 
work,  addressing  these  challenges  will  require  significant  innovation  over  the  next  few 
decades.  MDA/AS  has  been  charged  with,  among  other  things,  developing  innovative 
systems  and  concepts  to  the  point  where  MDA  leadership  can  make  informed  decisions 
about  transitioning  them  into  fonnal  development  programs.2 

This  chapter  outlines  three  types  of  recommended  efforts:  (1)  seeking,  refining 
and  developing  new  concepts  to  meet  various  mission  needs;  (2)  proving  out  promising 
solution  concepts  to  the  point  where  senior  MDA  management  can  reach  an  informed 
decision  about  whether  to  transition  them  into  a  formal  development  program;  (3) 
supporting  emerging  technologies  that  could  facilitate  breakthrough  improvements  in 
missile  defense  systems,  subsystems,  or  key  components,  and  in  doing  so  enable 
revolutionary  approaches  to  fundamental  missile  defense  challenges. 

PROBLEM  DESCRIPTION 

The  term  “innovation”  implies  the  development  of  a  new  product  or  process  that 
entails  a  departure  from  existing  approaches.  The  tenns  “disruptive  innovation”  or 
“radical  innovation”  have  been  used  to  denote  technologies  that  are  implemented  in  ways 
that  foster  profound  changes  in  operations  or  strategy,  as  opposed  to  an  “incremental” 
innovation  that  solves  a  narrower  problem — often  cost  or  feature  improvements — not 


http://www.acq.osd.mil/bmdo/bmdolink/html/mission.html. 

“MDA  examines  the  widest  possible  range  of  options,  potentially  inserting  enhancements  at  all  levels 
of  the  BMDS. . .including  evaluating  innovative  approaches,  paradigm  shifts,  and  new  concepts....” 
BMDS  Integrated  Program  Plan  (1 1  June  2003),  “Advanced  Systems”  section. 
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requiring  significant  adjustments  in  methods  or  approaches.  3  Radical  innovation  has 
been  defined  in  terms  of  the  type  of  new  performance  enabled,  that  is,  embodying  one  or 
more  of  the  following: 

•  New  to  the  world  performance  features. 

•  5-10  times  (or  greater)  performance  improvement. 

•  30-50%  (or  greater)  reduction  in  cost.4 

Radical  innovations  are  characterized  by  high  levels  of  not  only  technical  and 
market  uncertainty,  as  has  been  the  traditional  definition,  but  also  high  levels  of 
organizational  and  resource  uncertainty.  Organizational  uncertainties  refer  to  the  ability 
of  a  project  team  to  overcome  various  forms  of  internal  barriers  to  radical  change. 
Resource  uncertainties  refer  to  problems  in  obtaining  the  necessary  funding  and 
competencies  to  carry  radical  innovation  projects  forward.5  Indeed,  a  common  theme  in 
the  history  of  radical  innovation  is  that  the  originator  of  the  innovation  is  often  not  the 
same  organization  that  eventually  exploits  the  innovation  for  significant  gain.  Xerox  is 
infamous  for  inventing  but  not  exploiting  several  radical  innovations  in  the  modern 
personal  computer  industry,  such  as  the  graphical  user  interface,  the  first  word  processing 
program,  and  the  laser  printer.6 

The  radical  innovation  problem  could  be  considered  at  three  levels  in  MDA: 


3  The  concept  of  a  disruptive  technology/innovation  can  be  traced  back  to  Joseph  Schumpeter’s 
Capitalism,  Socialism  and  Democracy  (1942).  Schumpeter  describes  capitalist  economies  as  engines 
of  “creative  destruction”  in  which  new  firms  adopt  disruptive  innovations  that  challenge  existing 
firms’  dominance.  His  concept  was  based  on  recognition  that  long-term  profitability  in  a  competitive 
environment  depended  on  creating  market  inefficiencies  that  could  then  be  exploited.  Successful  firms 
make  above  average  profits  over  time  by  constantly  innovating,  that  is,  by  constantly  disrupting  the 
market.  More  recently,  the  term  “disruptive  technology”  was  popularized  in  Clayton  Christensen,  The 
Innovator’s  Dilemma:  When  New  Technologies  Cause  Great  Firms  to  Fail  (Harvard  Business  School 
Press,  1997).  He  defines  disruptive  technologies  as  ones  that  “bring  to  the  market  a  very  different 
value  proposition  than  had  been  available  previously.”  Geoffrey  Moore  uses  the  term  “discontinuous 
innovation”  in  Crossing  the  Chasm  (Harper  Business,  1991)  to  refer  to  “products  that  require  us  to 
change  our  current  mode  of  behavior  or  to  modify  other  products  and  services.” 

4  Richard  Leifer,  Christopher  M.  McDermott,  Gina  C.  O’Connor,  Lois  S.  Peters,  Mark  Rice,  and 
Robert  W.  Veryzer,  Radical  Innovation:  How  Mature  Companies  can  Outsmart  Upstarts ,  (Boston: 
Harvard  Business  School  Press,  2000). 

5  John  Doerr  of  Kleiner,  Perkins,  Caufield  &  Byers,  one  of  the  most  successful  and  respected  venture 
capitalists  in  history,  identifies  four  types  of  risk  similar  to  those  addressed  by  Leifer  et  al.  Doerr 
considers  financial,  technology,  market,  and  organization  risk  in  any  venture  investment  decision. 
Mitigation  of  these  risks  continues  to  guide  the  venture  development  process  after  investment.  Doerr’ s 
approach  to  risk  reflects  that  of  most  technology  venture  firms,  though  some  venture  firms  are  more 
successful  at  managing  these  risks  than  others. 

6  R.  C.  Alexander  and  D.  K.  Smith,  Fumbling  the  Future:  The  Story  of  Xerox  and  Personal  Computing 
(New  York:  Morrow,  1988). 
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•  Developing  missile  defense  capabilities  that  have  never  existed  before. 

•  Enabling  leap-ahead  performance  or  cost  improvements  in  missile  defense 
components. 

•  Seeking  and  supporting  revolutionary  technologies  and  “disruptive 
capabilities,”  that  is,  transformations  in  operations  and  strategy  typically  (but 
not  always)  enabled  by  synergistic  combinations  of  technologies. 

Current  MDA  programs  are  focused  almost  exclusively  on  employing  proven 
technologies  to  implement  concepts  that  are  believed  to  be  achievable  within  a  few  years. 
The  initial  system  being  deployed  at  Fort  Greeley,  Alaska,  is  seen  as  the  beginning  of  a 
spiral  development  program  in  which  future  capabilities  will  be  built  up  through 
incremental  improvements  in  system  components,  as  well  as  the  introduction  of  new, 
independent  layers  of  defense.  Because  current  missile  defense  capabilities  are 
“rudimentary,”7  any  future-oriented  MDA  program  is  aiming  to  create  something  that  has 
never  existed.  Future  layers  are  contemplating  leap-ahead  improvements  in  critical 
components  (e.g.,  much  more  powerful  and  cost-effective  lasers). 

Our  focus  here  will  be  on  the  conception,  selection,  management,  and  transition 
of  MDA  programs  aimed  at  fostering  revolutionary  technologies  and  disruptive 
capabilities  exceeding  what  can  be  achieved  through  incremental  improvements  of 
current  missile  defense  approaches.  Our  previous  work  for  MDA8  found  that  encouraging 
radical  innovation  requires  different  management  processes  and  technology  competencies 
than  are  typically  employed  in  both  government  and  industry  S&T9  programs.  In  many 
cases,  corporations  and  government  programs  would  create  a  separate  “organization  to 
support  long-term  S&T  and  foster  radical  innovation  that  is  independent  of. ..project 
offices,  but  works  with  them  for  implementation.”10  For  consistency,  the  structures  and 
processes  described  in  this  chapter  will  adopt  the  framework  in  Chapter  I,  in  which  an 
MDA  innovation  organization  would  operate  as  an  independent  group  headed  by  a  Chief 
Innovation  Executive  but  embedded  within  the  overall  MDA  S&T  program  under  the 
MDA/AS.  However,  these  functions  could  be  implemented  in  other  ways.  The  manner 
and  extent  of  an  innovation  group’s  integration  within  MDA  would  depend  on  the  details 


7  Secretary  of  Defense  Donald  Rumsfeld,  Ronald  Reagan  Public  Policy  Briefing,  12  October  2003, 
http://www.defenselink.mil/transcripts/2003/tr2003 10 10-secdef0752.html. 

8  Van  Atta  et  al.  Science  and  Technology’. 

9  “Science  and  technology”  is  the  U.S.  government  terminology  for  what  in  the  commercial  world  is 
typically  called  research  and  development  (R&D). 

10  Van  Atta  et  al.  Science  and  Technology’,  p.  ES-2. 
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of  current  MDA  organizations  and  processes,  as  well  as  on  MDA  leadership  priorities. 
Although  these  issues  were  not  engaged  in  detail  by  this  study,  certain  general 
implementation  issues  are  discussed  in  the  following  section. 

MDA  IMPLEMENTATION  ENVIRONMENT 

A  successful  missile  defense  system  will  consist  of  a  complex,  integrated  “system 
of  systems”  in  which  multiple  layers  of  integration  will  be  required.  Building  such  a 
system  of  systems  entails  large-scale  engineering  work  that  requires  solving  thousands  of 
problems  involving  tradeoffs  among  technical  properties,  manufacturing  limitations  and 
costs,  and  customer  values.  That  engineering  work  will  typically  reveal  gaps  in  the 
system  concept  requiring  adjustment  of  subsystem  specifications,  interfaces,  and 
performance  margins.  Sometimes  the  process  will  also  reveal  gaps  in  basic  technologies 
and  even  scientific  understandings.  This  is  particularly  true  in  the  missile  defense 
domain,  where  the  speed  and  energy  of  certain  phenomena  present  problems  well  beyond 
the  state  of  the  art  in  key  technologies;  in  some  cases,  the  underlying  physics  may  not  be 
thoroughly  understood. 

Scientific  and  engineering  approaches  to  missile  defense  trace  back  to  antiaircraft 
systems  developed  during  and  after  World  War  II.  The  United  States  built  a  variety  of 
missile  defense  systems  during  those  years,  culminating  in  the  Safeguard  System,  which 
was  eventually  deactivated  in  1976.  Meanwhile,  the  DEFENDER  Program,  begun  by  the 
Advanced  Research  Projects  Agency  in  1959,  was  the  beginning  of  a  series  of  scientific 
and  engineering  projects  focused  on  understanding  and  measuring  the  basic 
phenomenology  of  missile  launch,  ballistic  flight,  and  atmospheric  reentry.11  The  MDA 
of  today  is  the  latest  incarnation  in  a  series  of  high-level  management  organizations 
created  since  1983  within  OSD  and  intended  to  focus  overall  Department  of  Defense 
(DoD)  efforts  to  build  missile  defenses. 

OSD  missile  defense  organizations  have  varied  significantly  over  the  years  with 
changes  in  the  perceived  threat  environment  and  differing  emphases  and  priorities  of 
different  administrations.  This  history  is  relevant  to  the  innovation  problem  to  the  extent 
that  it  has  created  (1)  a  set  of  missile  defense  concepts,  technologies,  and  science  that 
have  been  investigated  to  various  levels;  (2)  experienced  personnel  and  centers  of 
excellence,  mostly  outside  of  MDA;  and  (3)  expectations  about  future  needs  and 


1 1  Richard  Van  Atta  and  Jack  Nunn,  “DEFENDER — Science  and  Technology  for  Ballistic  Missile 
Defense,”  Chapter  XI  in  Van  Atta  et  ah,  Science  and  Technology’. 
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priorities,  given  that  the  missile  defense  problem  will  never  be  completely  “solved.” 
Adversaries  will  respond  to  U.S.  missile  defense  developments,  presenting  an  ever- 
changing  series  of  threats  and  challenges.  Moreover,  the  time  required  to  develop  and 
deploy  a  robust  missile  defense  capability  is  much  longer  than  the  tenure  of  any  given 
administration. 

The  Bush  Administration  made  clear  its  intention  to  deploy  national  missile 
defenses.  The  United  States  gave  notice  on  13  December  2001  that  it  would  withdraw 
from  the  Anti-Ballistic  Missile  Treaty,12  and  on  2  January  2002,  the  Secretary  of  Defense 
signed  a  memorandum  creating  MDA  with  a  charge  to  “field  elements  of  the  overall 
BMDS  as  soon  as  practicable.”  Consistent  with  this  charge,  MDA  has  been  focused  on 
deployment  of  a  BMDS  with  limited  capabilities,  with  relatively  few  resources  for  or 
attention  to  longer  term,  innovative  approaches13  or  accessing  independent  technology 
identification  and  assessment.14  (Various  outside  groups — most  notably  the  Defense 
Science  Board — have  continued  to  examine  BMDS  technologies  and  propose  alternative 
system  concepts.)  Although  MDA’s  adoption  of  a  spiral  development  philosophy  has 
created  a  focus  on  incremental  improvements,  MDA  planning  documents  call  for 
encouragement  of  innovation,  and  some  processes  appear  to  be  in  place  to  support  it.15 

IMPLEMENTATION  OVERVIEW 

Design  Principles 

The  essence  of  innovation  is  the  conception,  refinement  and  realization  of  “new 
combinations...  something  newly  tried.”16  In  the  case  of  technology-based  innovation,  an 
unmet  or  under-met  market/mission  need  is  linked  (combined)  with  a  set  of  technologies 
that  satisfies  an  important  part  of  the  need.  For  MDA,  mission  needs  are  subtasks  (e.g., 
launch  detection)  aimed  at  meeting  broad  missile  defense  challenges  (e.g.,  boost  phase 
defeat).  Promoting  innovation  in  MDA  will  require  three  types  of  efforts:  (1)  seeking, 
refining  and  developing  new  concepts  to  meet  various  mission  needs;  (2)  proving  out 


12  “President  Discusses  National  Missile  Defense,”  http://www.whitehouse.gov/news/releases/ 
2001/12/2001 1213-4.html. 

13  Randy  Barrett,  “Critics  Question  MDA’s  Interest  in  New  Technology,”  Space  News,  3  February  2003. 

14  Randy  Barrett,  “Pentagon  Backpedals  on  Schedule  for  Space-Based  Missile  Interceptors,”  Space 
News,  7  July  2003. 

13  BMDS  Integrated  Program  Plan,  11  June  2003. 

16  J.  A.  Schumpeter,  The  Theory  of  Economic  Development  (Cambridge,  MA:  Harvard  University  Press, 
1934). 
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promising  solution  concepts  to  the  point  where  senior  MDA  management  can  reach  an 
informed  decision  about  whether  to  transition  them  into  a  formal  development  program; 
and  (3)  supporting  emerging  technologies  that  could  facilitate  breakthrough 
improvements  in  missile  defense  systems,  subsystems,  or  key  components  and  in  doing 
so  enable  revolutionary  approaches  to  fundamental  missile  defense  challenges. 

There  are  many  ways  to  realize  these  objectives.  In  the  current  implementation 
environment,  the  design  principles  in  Figure  II- 1  guided  our  articulation  of  options  for 
MDA.  The  strategy  principles  mean  that  innovative  technology  projects  must  not  be 
“science  for  science’s  sake”  but  rather  technology  development  aimed  at  meeting  mission 
objectives.  The  structure  principles  mean  that  mechanisms  for  promoting  innovation 
should  draw  from  many  different  sources — including  sources  outside  MDA — and  be 
flexible  and  adaptable  to  the  particular  skills  and  personalities  of  the  individuals 
involved.17  The  process  principles  mean  that,  although  flexible,  the  process  must  also  be 
disciplined  so  projects  are  steered  toward  mission  objectives;  in  other  words,  it  is  not 
about  just  “casting  a  thousand  seeds”  and  seeing  what  grows.18  Finally,  the 
implementation  principles  mean  the  process  should  be  implemented  in  small  steps, 
growing  organically  as  it  begins  to  register  successes  and  gain  organizational  credibility. 


Strategy  Mission  Focus 

Senior  Strategic  Leadership 
Alignment  with  MDA 


Structure  Scaleable  ‘Cell’  Structure 
Cross-functional  Teams 
External  Network  Focus 


Process  Stage-Gate  Management  Process 
Explicit  Support  Processes 
Process  Integration  with  MDA 

Implementation  Phased  Implementation 

Start  small  and  scale  up 
Organic  Growth  (vs.  pre-planned) 

Figure  11-1.  Design  Principles  for  an  MDA  Innovation  Organization 


17  Chapter  IV  discusses  various  networking  and  outreach  mechanisms  appropriate  to  the  MDA  context. 

18  Chapter  V  delves  in  detail  into  a  variety  of  analytic  tools  that  can  facilitate  a  disciplined  project  and 
portfolio  management  approach. 
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Process  Overview 


The  envisioned  innovation  promotion  process  has  two  main  elements: 

•  Initial  Concept  Creation — The  proposed  structures  and  processes  are  designed 
to  originate  and  articulate  new  concepts  by  helping  people  regularly  make 
connections  between  unmet  or  undermet  mission  needs  and  appropriate 
technologies  or  systems.  New  combinations  of  mission  needs  with 
technologies  or  systems — concepts — must  then  be  sufficiently  refined  to  be 
considered  as  candidate  S&T  projects.  The  resulting  S&T  candidates  could  be 
aimed  at  innovations  in  enabling  subsystems  or  technologies,  or  they  could  be 
proposals  for  large-scale  projects,  building  an  integrated  capability  from  more 
mature  technologies. 

•  Refinement  and  Development — Following  the  initial  conception  stage,  a 
structured  process  should  be  employed  to  winnow  out  S&T  candidates  through 
increasingly  rigorous  review.19  If  a  project  survives  to  prototype 
demonstration,  it  may  be  considered  for  transition  into  either  a  formal 
development  program  (if  it  is  judged  to  provide  sufficient  value  as  an  acquired 
and  deployed  capability)  or  a  radical  innovation  technology  project  (if  it 
represents  a  potential  enabling  technology). 

These  processes  must  not  be  rigid  or  linear.  Initial  concept  creation,  an  inherently 
creative  process,  is  often  referred  to  as  the  “fuzzy  front  end.”20  However,  this  tenn  has 
been  used  less  frequently  as  researchers  and  practitioners  have  come  to  understand  key 
elements  of  front  end  processes.21  These  processes  can  be  systematically  encouraged, 
though  not  inflexibly  controlled.  Similarly,  during  refinement  and  development,  a 
concept  that  fails  to  pass  through  a  particular  stage-gate  review  may  be  moved  back  to  an 
earlier  stage  for  development  in  a  different  direction.  Furthermore,  technology  stage-gate 
criteria  should  not  be  as  demanding  as  a  typical  “business  case”  analysis  that  would  be 
applied  to  the  development  of  incremental  innovations.  The  criteria  applied  at  each  stage 
should  differ  depending  on  the  nature  of  the  uncertainties  to  be  resolved — technical, 
mission,  organizational,  and  resource — and  the  anticipated  time  horizon  of  the  concept 


19  Chapter  V  provides  a  detailed  discussion  of  the  technology  stage  gate  management  process. 

20  The  term  was  coined  by  Preston  Smith  and  Donald  Reinertsen,  Developing  Products  in  Half  the  Time, 
(Van  Nostrand  Reinhold,  1991). 

21  Peter  A.  Koen,  Greg  M.  Ajamian,  Scott  Boyce,  Allen  Clamen,  Eden  Fisher,  Stavros  Fountoulakis, 
Albert  Johnson,  Pushpinder  Puri  and  Rebecca  Seiber,  “Fuzzy  Front  End:  Effective  Methods, Tools  and 
Techniques,”  in  Paul  Belliveau,  Abbie  Griffin,  and  Stephen  Somermeyer  (eds.),  The  PDMA  ToolBook 
for  New  Product  Development  (John  Wiley  &  Sons,  5  April  2002).  See  also  Jongbae  Kim,  and  David 
Wilemon,  “Focusing  the  Fuzzy  Front-end  in  New  Product  Development,”  R&D  Management,  Vol.  32, 
pp.  269-279,  2002. 
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(which  has  an  impact  on  these  uncertainties).  For  instance,  a  concept  may  have  clear 
application  in  a  missile  defense  system  while  also  presenting  significant  technical 
uncertainty.  Its  refinement  path  would  focus  on  characterizing  and  managing  this 
technical  uncertainty.  If  the  concept  were  to  transition  into  a  fonnal  project,  that  project 
would  be  managed  in  a  manner  typical  of  technology  development  programs.  Another 
concept  might  involve  a  novel  application  of  well-understood  technologies  that  would,  if 
successful,  replace  an  existing  program.  Its  refinement  path  would  depend  on  measures 
aimed  at  building  a  constituency  to  overcome  organizational  resistance.  If  successful,  the 
ultimate  program  would  be  managed  in  a  manner  typical  of  systems  development 
program. 

Appropriate  processes  can  play  a  critical  role  in  creating  or  discouraging  a 
“culture”  of  innovation  within  an  organization.  For  instance,  effective  networking 
regimes  with  experts  external  to  MDA  can  help  overcome  the  well-known  “not  invented 
here”  syndrome — a  disposition  against  externally  generated  concepts  and  technologies. 
Combined  with  effective  incentive  regimes,  these  processes  can  encourage  sensible  risk 
taking  as  well  as  enable  innovation  teams  to  shelve  less  promising  projects. 

Key  Functions 

The  process  described  above  at  least  five  functions  that  need  to  be  addressed  in  an 
organization  designed  to  promote  radical  innovation: 

•  Mission  needs  discovery  and  definition. 

•  Concept  development  and  refinement. 

•  Technology  identification  and  assessment. 

•  Technology  creation. 

•  Proof-of-concept  experimentation  and  demonstration. 

Mission  needs  discovery  and  definition 

To  find  useful  solutions,  it  is  important  to  understand  which  problems  present  the 
most  daunting  challenges,  as  well  as  the  highest  impact  opportunities.  This  requires 
reasonable  flexibility  and  adaptability,  so  that  the  effort  to  gather  needs  does  not  become 
prematurely  wedded  to  a  set  of  predefined  requirements  from  existing  users,  making 
consideration  of  new  approaches  difficult.  The  articulation  of  needs  should  lead  to  brief, 
overall  statements  of  desired  capabilities  that  have  few,  if  any,  technical  parameters. 
Stating  needs  only  in  terms  of  capabilities  already  known  to  exist  inhibits  innovative 
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concept  creation.  In  general,  existing  users  are  not  good  at  thinking  beyond  their  current 
needs,  and  perfonnance  objectives  for  programs  in  current  development  tend  to  be  stated 
in  the  context  of  existing  solutions.  The  means  for  accomplishing  a  particular  objective 
tend  to  be  visualized  in  terms  of  improvements  to  the  existing  technology  or  capability, 
rather  than  a  brand  new  solution. 

Thinking  beyond  the  limits  of  existing  processes  and  biases  requires  a  particular 
type  of  observer.  Individuals  released  from  the  status  quo  solution  have  a  much  greater 
opportunity  of  discovering  new  solutions,  but  they  also  need  to  know  what  to  be  aware 
of.  One  effective  approach  is  to  restate  mission  needs  in  tenns  of  a  detailed  articulation  of 
subsystem  and  capability  needs,  then  match  these  needs  with  general  scientific  and 
technological  arenas  that  might  offer  solutions.  Innovation  teams  would  then  have 
specific  needs  in  mind  while  investigating  emerging  S&T.  Thinking  of  needs  concurrent 
with  the  technology  search  is  critical  for  effective,  sustainable  concept  development. 
Needs  identification  and  awareness  is  a  strategy  under-exploited  by  many  cutting  edge 
technology  development  groups,  from  industry  to  government  and  academic  labs. 

Concept  development  and  refinement 

The  creative  process  of  concept  development  is  difficult  to  capture.  The  essence 
is  fostering  particular  types  of  interactions  between  people  familiar  with  needs  and 
applications  and  those  familiar  with  technologies  and  capabilities.  For  example,  in  the 
Technology  Assessment  Pilot  Project  described  briefly  in  Chapter  I,  a  combination  of 
people  experienced  with  missile  defense  problems  interacted  with  technologists.  Working 
together,  they  undertook  a  systematic,  top-down  analysis  that  proceeded  from  high-level 
BMDS  challenges,  to  novel  operational  concepts  for  addressing  these  challenges,  to 
systems  description  (at  a  high  level)  that  might  realize  the  functions,  to  technical 
capabilities  that  could  enable  such  systems  to  be  built.  A  number  of  creativity  techniques 
can  be  employed  to  encourage  novel  thinking  from  users,  suppliers,  scientists,  and 
technologists. 

Innovative  concepts  can  also  arise  during  the  engineering  work  involved  in 
building  a  system  of  systems.  Such  work  will  sometimes  reveal  gaps  in  basic 
technologies  and  even  in  scientific  knowledge.  Truly  innovative  organizations  encourage 
and  exploit  the  interaction  between  application-oriented  system  designers  and 
technology-oriented  scientists  and  engineers.  Faced  with  a  technical  roadblock  in  systems 
development,  scientists  may  be  called  in  to  search  for  new  information,  models,  and 
tools.  In  doing  so,  systems  assumptions  may  be  challenged  and  new  understanding 
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generated,  leading  to  consideration  of  innovative  solutions.22  The  original  problem  is 
transfonned  by  a  deeper  understanding  of  the  context,  leading  to  novel  solution  paths. 

Technology  identification  and  assessment 

Technology  identification  and  assessment  requires  going  from  a  general  statement 
of  need  to  one  with  sufficient  definition  to  provide  guidance  to  those  charged  with 
worldwide  technology  scouting.  This  means  translating  a  general  need  to  a  clearly 
defined  set  of  key  performance  parameters  required  to  achieve  desired  application  results. 
Performance  metrics  must  be  defined  and  documented  so  that  discovered  technologies 
can  be  adequately  assessed  relative  to  mission  needs.23  In  the  early  stages  of  concept 
development,  performance  metrics  should  be  broad,  to  encourage  consideration  of  a  wide 
range  of  solutions,  as  it  is  rarely  possible  to  know  what  solutions  will  become  available. 
As  concepts  become  more  defined,  performance  criteria  and  metrics  should  become  more 
specific.  At  the  same  time,  knowledge  of  available  capabilities  influences  definitions  of 
objectives.  Through  an  iterative  process,  definition  of  mission  needs  and  technology 
metrics  co-evolve. 

Technology  creation 

As  part  of  its  innovation  process,  MDA  will  not  be  able  to  rely  solely  on 
technologies  developed  by  others.  The  development  of  new  concepts  will  foster  ideas  for 
new  technology  investigations  that  may  diverge  from  what  is  being  pursued  elsewhere, 
given  the  unique  issues  associated  with  missile  defense.  The  technology  creation  effort 
would  be  focused  on  funding  potentially  revolutionary  technologies  with  potential  for 
broad  application  to  missile  defense  challenges,  but  that  are  too  immature  to  be  linked  to 
specific  top-down  requirements  of  a  single  program.  U.S.  Government  experience  with 
commercial  technology  programs  in  the  past  decade  has  demonstrated  that  to  access  and 
leverage  certain  emerging  technology  developments,  MDA  will  need  to  monitor  and 
shape  these  technologies  in  the  early  stages.24  As  an  example,  during  the  1970s, 
DARPA’s  steady,  forward-looking  promotion  of  critical  technologies — before  their 
national  security  significance  became  clear — supported  U.S.  dominance  of  entirely  new 


22  Interview  with  Bradley  Hartfield,  10  June  2002. 

23  This  same  type  of  refinement  is  important  for  the  management  of  long-term  strategic  technologies,  as 
described  in  Chapter  IV. 

24  Report  of  the  Defense  Science  Board  Task  Force  on  the  Technology  Capabilities  of  Non-DoD 
Providers  (Washington,  D.C.:  Office  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology 
and  Logistics,  June  2000). 
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industries,  the  technologies  from  which  underlay  the  superior  capabilities  of  many 
current  U.S.  military  systems.25  Chapter  IV  discusses  various  ways  for  MDA  to 
structuring  collaborative  research  programs  with  other  government  entities,  commercial 
companies,  and  university  programs. 

Proof-of-concept  experimentation  and  demonstration 

Some  concepts  will  evolve  to  the  point  where  they  can  be  prototyped  to 
demonstrate  concept  feasibility  in  increasingly  realistic  application  environments. 
Demonstration  is  often  a  critical  prerequisite  to  garnering  sufficient  political  support  for 
what  can  be  relatively  expensive  development  efforts.  Before  such  demonstrations, 
significant  system-of-systems  integration  work  must  be  accomplished,  including  complex 
experimentation  and  assessments.  As  such,  S&T  organizations  need  to  employ  careful 
selection,  planning,  and  budgeting  mechanisms  for  demonstrations.  Different  personnel 
and  management  capabilities  are  required  for  large-scale  integration  and  experimentation 
projects  than  are  used  for  exploratory  concept  development  and  refinement.  The  scale 
and  scope  of  such  experiments  can  drain  resources  from  earlier  stage  exploratory  S&T. 
Further,  projects  that  have  progressed  to  this  stage  will  engender  the  interests  of  defense 
contractors,  inviting  use  of  the  political  maneuvering  to  affect  choices. 

These  basic  functions — needs  discovery,  concept  development,  technology 
assessment,  technology  creation,  and  system  integration/experimentation — suggest  the 
need  for  various  supporting  processes,  including: 

•  Networking  and  outreach — Chapter  IV  discusses  various  networking  and 
outreach  mechanisms  appropriate  to  the  MDA  context. 

•  Project  and  process  management — Beyond  leadership,  new  concept 
development,  as  with  any  development  project,  requires  documentation, 
coordination,  and  measured  bureaucratic  control. 

•  Overall  portfolio  coordination  and  leadership — As  the  number  of  innovation 
projects  increases,  direction  will  become  increasingly  important,  especially  to 
ensure  that  sufficient  efforts  are  devoted  to  higher  risk,  higher-payoff  efforts. 


25  Richard  Van  Atta  and  Michael  Lippitz,  Transformation  as  Transition:  DARPA ’s  Role  in  Fostering  an 
Emerging  Revolution  in  Military’  Affairs ,  IDA  Paper  P-3698  (Alexandria,  Va.:  Institute  for  Defense 
Analyses,  April  2003). 
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Team  Composition 

As  suggested  above,  innovative  concepts  typically  arise  as  applications-oriented 
and  technology-oriented  people  converge  on  mutually  interesting  problems  in  a  process 
that  draws  upon  both  types  of  skills  and  temperament.  Well-designed  processes  enable 
people  to  be  vastly  more  effective  and  create  a  productive  “culture”  of  innovation  within 
an  organization.  However,  no  process  can  overcome  poor  staffing,  particularly  if  team 
members’  skills  and  dispositions  are  poorly  matched  to  innovation  tasks.  Even  good 
people  can  be  kept  from  accomplishing  their  objectives  by  inadequate  leadership  and 
insufficient  contact  with  their  broader  communities  of  practice.26 

Innovation  leadership  is  of  particular  importance  for  accessing  resources, 
motivating  players  inside  and  outside  a  team,  integrating  divergent  views,  and  interacting 
with  senior  executives.  Innovation  leaders — hereafter  referred  to  as  “Concept 
Managers” — must  be  able  to  articulate  the  promise  of  a  concept  and  be  respected  enough 
technically  to  garner  top-level  support.  Such  highly  skilled,  synthetic  thinkers  are  a  rare 
breed,  as  are  top  executives  open  minded  and  courageous  enough  to  facilitate  them.  In 
the  specialized  MDA  community,  it  may  be  particularly  difficult  to  find  individuals  who 
have  sufficient  experience  in  missile  defense  to  be  technically  credible  while  also 
exhibiting  such  entrepreneurial  qualities.  Even  if  MDA  had  such  individuals,  it  would 
still  gain  from  the  perspectives  of  outside  talent.  As  a  result,  a  portion  of  those  recruited 
as  Concept  Managers  should  be  from  outside  the  current  MDA  environment.  Ultimately, 
the  MDA  innovation  program  should  find  a  balance  between  internal  “veterans”  and 
external  individuals,  each  with  an  appropriate  combination  of  technological  expertise  and 
innovation  capabilities. 

An  organizational  strategy  encouraging  both  internal  and  external  networking  and 
collaboration  nurtures  the  evolution  of  an  innovative  organization.27  An  innovation  team 
engaged  with  external  knowledge  sources  and  collaborators  will  be  much  more  likely  to 
be  exposed  to  new  approaches  to  problems.  MDA  innovation  management  should 
support  the  search  for  the  best  sources  for  applicable  knowledge  wherever  it  resides.  It 
should  also  work  to  develop  and  promulgate  shared  understandings  of  alternative  system 
architectures  and  the  particular  challenges  that  flow  from  them — that  is,  a  common 


26  Alistair  Cockburn,  “Agile  Software  Development  2:  The  People  Factor,” 

http://alistair.cockbum.us/crvstal/articles/asdpf/asd2peoplefactor.htm  (accessed  10  November  2003). 

27  See  Chapter  IV  for  further  discussion  of  networking  and  outreach  mechanisms  appropriate  to  the 
MDA  context. 
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language  to  express  “grand  challenges.”  Shared  understandings  are  important  for  the 
conception  of  innovative  solutions  in  missile  defense,  given  the  high  level  of  systems 
integration  required. 

A  PHASED  IMPLEMENTATION  APPROACH 

Activities  aimed  at  promoting  innovation  can  be  implemented  by  MDA  in 
limited,  scalable  phases,  beginning  with  a  small  staff  that  could  grow  as  promising 
concepts  arise.  For  descriptive  purposes,  the  growth  process  is  separated  into  four 
conceptual  phases. 

Phase  1  Concept  Managers 

Phase  2  Supported  Concept  Managers 

Phase  3  Supported  Concept  Managers  with  Infrastructure 

Phase  4  The  Application/Technology  Matrix  Organization 


Review  of  Phased  Implementation  of  Concept  Development 

The  four  phases  described  above  envision  MDA  concept-development  activities 
growing  from  a  small  set  of  individuals  responsible  for  all  aspects  of  the  innovation 
promotion  process,  to  a  formal,  matrixed  application  and  technology  organization  akin  to 
what  is  found  in  some  well-established  industrial  organizations.  The  idea,  depicted 
Figure  II-2,  is  to  grow  organically,  from  individual  Concept  Managers  to  cross-functional 
interacting  teams  of  applications-focused  and  technology-focused  people.  Because 
functions  will  not  be  rigidly  defined,  personnel  can  move  easily  among  roles  and  share 
ideas  across  teams. 


expert  5.  needs  differentiation  make  strong  connections  between  similar 

3.  group  developer  functions  on  each  team  (e.g.-  group 

leader  6.  Technology  leaders,  technology  experts) 

seeker 

Figure  11-2.  The  Evolution  of  Innovation  Teams  and  Other  Internal  Networks 
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The  growth  path  will  depend  on  the  skills,  dispositions,  and  interests  of  the  people 
involved.  For  the  purposes  of  discussion,  particular  functions  and  support  processes  are 
described  as  though  they  were  the  domain  of  individual  specialists,  but  this  may  not  be 
the  case.  Individuals  tend  to  complement  and  reinforce  each  other,  with  specialization 
evolving  naturally  based  on  the  strengths  and  weaknesses  of  particular  team  members. 
Executive  leadership  and  support  includes  composing  and  developing  effective  cross¬ 
functional  teams. 

Phase  1  Implementation:  Concept  Managers 

Concept  development  is  the  core  of  the  innovation  promotion  process.  New 
concepts  arise  in  many  ways  and  can  encompass  enabling  technologies,  innovations  in 
subsystems,  or  broad  new  approaches  to  fundamental  missile  defense  challenges.  At 
minimum,  an  organization  can  enhance  new  concept  development  by  protecting  and 
facilitating  the  highly  skilled,  synthetic  thinkers  who  generate  new  ideas.  Concept 
Managers  would  focus  on  developing  solution  concepts,  bringing  together  technologists, 
integrators,  and  operational  experts  from  a  variety  of  stakeholder  organizations,  including 
potential  commercial  partners.  For  example,  DARPA  continually  recruits  technically 
skilled  people  with  innovative  ideas — and  the  drive  and  disposition  to  make  them  real. 
DARPA  program  managers  are  conferred  independence,  responsibility,  and 
accountability  and  are  expected  to  show  results  in  3  to  4  years.  DARPA  is  one  of  the 
DoD  organizations  that  provides  a  useful  point  of  comparison  because  it  has  been  able  to 
implement  some  of  the  best  practices  identified  in  the  first  phase  study — consistent,  long¬ 
term  support  of  critical  technologies  and  simultaneous  management  of  top-down  and 
bottom-up  concepts. 

As  a  first  step,  MDA  could  recruit  or  designate  Concept  Managers  and  provide 
them  with  sufficient  funds  to  contract  externally  (where  necessary)  for  support  in  mission 
needs  discovery  and  definition,  technology  identification  and  assessment,  networking  and 
outreach,  and  project  and  process  management.  Examples  of  part-time  resources  that 
could  be  useful  to  a  Concept  Manager  include  the  following: 

•  A  pool  of  technical  capabilities  to  provide  occasional  “bench  scientific 
support”  to  test  out  new  subsystem  and  undertake  “due  diligence”  on  the 
technology  components  of  new  concepts. 
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•  Experts  in  scenario  building,  wargaming,  and  the  like,  to  help  develop 

28 

innovative  approaches  to  meet  strategic  needs. 

•  Analytic  expertise — especially  modeling  and  simulation  capability — to 
facilitate  early,  inexpensive  testing  of  critical  parts  of  new  concepts. 

•  Technology  forecasting  experts  (mostly  outside  MDA  to  avoid  potential 
perceived  conflicts  of  interest). 

In  addition,  many  management  consulting  firms  provide  services  to  clients  explicitly 
designed  to  facilitate  creative  new  concept  development,  through  personnel  creativity 
training,  the  creation  of  new  organizational  environments,  outside  design  specialist 
services,  and  process  analyses.29 

Concept  Managers  could  be  recruited  from  among  the  large  group  of  people  with 
government  missile  defense  experience,  as  well  as  the  technical  experts  at  national 
laboratories  and  academia  with  relevant  applications  background.  Concept  Managers 
must  be  technically  capable  to  provide  credible  leadership  and  contributions  in  the  eyes 
of  technologists.  An  insufficient  technical  aptitude  will  impede  the  leader’s  ability  to 
pose  the  right  questions.  The  leader’s  reputation  and  capabilities  must  also  confer 
credibility  on  the  innovation  team. 

The  success  of  this  model  depends  to  a  large  degree  on  the  insight  and 
imagination  of  Concept  Managers,  the  coherence  of  the  strategic  direction  from  senior 
leadership,  and  the  Concept  Managers’  ability  to  integrate  the  potential  “customer’s” 
mission  needs  into  project  direction.  Absent  strategic  direction  and  appropriate  (though 
light-handed)  oversight,  this  approach  can  devolve  into  a  “cast  a  thousand  seeds” 
enterprise.  Hence,  as  suggested  in  Chapter  I,  a  Chief  Innovation  Executive  should  be 
appointed  or  designated  to  oversee  the  work  of  MDA  Concept  Managers. 

Phase  2  Implementation:  Supported  Concept  Managers 

As  Phase  1  effort  gains  credibility — or  if  MDA  decides  to  take  a  more  aggressive 
initial  approach — MDA  could  add  additional  Concept  Managers  and  provide  internal 
support  staff  or  a  stable  outsourced  capability  to  help  them  identify  and  recognize 


The  development  of  detailed  scenarios  can  be  particularly  important  for  refining  technology  needs. 
See,  for  instance,  R.  Van  Atta  and  M.  Lippitz,  Transition  and  Transformation:  DARPA ’s  Role  in 
Fostering  the  Emerging  Revolution  in  Military’  Affairs,  Volume  I — Overall  Assessment,  IDA  Paper  P- 
3698  (Alexandria,  Va.:  Institute  for  Defense  Analyses,  April  2003). 

29  Martin  Hyatt,  Metaphoric  Models  of  Creative  Thinking,  Ph.D.  Dissertation,  Stanford  University 
Department  of  Management  Science  and  Engineering,  1999. 
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opportunities,  as  well  as  encourage  recruitment  of  new  Concept  Managers.  These  support 
staff  or  external  capabilities  would  systematically  pursue  mission  needs  discovery  and 
definition,  provide  continuous  technology  monitoring  and  assessment,  build  and  maintain 
a  network  of  technical  and  application  expertise  relevant  to  MDA  strategic  challenges, 
perform  regular  outreach  in  important  technology  communities,  and  provide  professional 
support  for  project  and  process  management.  Concept  managers  and  support  staff  could 
form  into  cross-functional  teams  with  complementary  skills  and  dispositions  (with 
specialization  arising  and  evolving  naturally,  based  on  the  strengths  and  weaknesses  of 
particular  members). 

Recent  research  suggests  that  Concept  Managers  would  benefit  from  two 
particular  types  of  support:  (1)  “hunters,”  who  actively  seek  out  ideas  with  application 
potential,  and  (2)  “gatherers,”  who  understand  strategic  needs  and  are  poised  to  recognize 
and  technically  validate  promising  new  ideas.30 

[Hunters  are  people  with]  technical  training,  but  they  are  more  likely  to  be 
experienced  in  marketing  or  business  development  (in  an  industrial 
environment)  or  in  high-level  systems  management  (in  government). 

Perhaps  more  importantly,  a  successful  hunter  knows  how  to  articulate  the 
opportunity  in  compelling  terms  that  gain  the  attention  of  higher 
management — something  that  few  bench  scientists  are  skilled  at 
doing... [Gatherers,  on  the  other  hand,]  have  the  technical  sophistication  to 
assess  what  they  encounter.  In  addition,  their  life  experiences  have 
engendered  a  certain... awareness  of  markets  and  social  and  scientific 
trends... first-line  and  midlevel  research  managers  and  senior  scientists 
(often  play)  the  role  of  gatherer.31 

In  the  MDA  context,  hunters  could  be  thought  of  as  “Mission  Analysts.”  The  key 
skills  involved  are  (1)  perceptive  listening,  (2)  a  talent  for  seeing  through  immediate 
problems  to  broader  needs,  and  (3)  the  ability  to  delineate  needs  from  multiple 
perspectives  and  in  sufficient  detail  to  provide  guidance  to  technologists  and  developers. 
They  would  likely  come  from  within  MDA  staff  or  a  government  laboratory.  Operational 
experts  who  understand  current  applications  and  Service  cultures  could  help  focus  teams 
on  areas  likely  to  have  value  for  end  users. 

Senior  technologists  could  serve  as  gatherers.  Ideally,  these  are  respected,  well- 
connected  people  with  a  reputation  as  contributors  to  missile  defense  or  related 
applications.  They  would  be  network  builders,  who  would  provide  access  to  technical 


30  Leifer  et  al..  Radical  Innovation. 

31  Ibid.  p.  11-15. 
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evaluators,  and  attract  top  talent  to  participate  on  innovation  teams.  These  players  could 
come  out  of  program  offices  or  laboratories.  A  related  support  function  would  be 
“Technology  Scouts.”  These  are  experts  in  technology  search  who  vigorously  pursue 
learning  about  new  technologies  and  their  possibilities.  People  with  this  orientation  are 
hard  to  find.  They  are  similar  in  many  respects  to  Concept  Managers,  but  without 
necessarily  possessing  the  requisite  leadership  and  management  skills.  These  players  can 
be  important  contributors  to  concept  development  and  refinement,  as  well  as  technology 
forecasting. 

By  this  phase,  it  will  be  important  to  begin  fonnalizing  the  technology  stage-gate 
process  for  project  evaluation.  The  stage-gates  must  be  defined  to  assess  projects  in  terms 
of  strategic  fit;  potential  impact;  and  technical,  mission,  organizational,  and  resource 
risks.  One  of  the  most  important  metrics  should  be  “learning  per  dollar.”32  That  is,  how 
can  evaluations  be  accomplished  efficiently,  to  bring  focus  to  the  most  promising  ideas 
(and  shelving  of  others  for  future  consideration)?  Assessment  of  strategic  fit  would 
include  consideration  of  MDA,  OSD,  and  Service  objectives,  as  well  as  synergies  and 
conflicts  with  existing  programs.  A  conflict  would  not  necessarily  eliminate  a  project 
from  consideration  but  would  suggest  the  need  for  different  types  of  management.  The 
stage-gate  process  itself  would  need  to  be  developed  by  a  cross-functional  team  with  a 
variety  of  experience,  and  tested  against  some  past  or  present  sample  projects  for 
practicality.33 

The  support  functions  outlined  here  are  only  illustrative  of  useful  capabilities  in 
an  innovation  team.  Not  all  of  these  functions  would  necessarily  need  to  be  staffed.  One 
would  expect  team  members  to  play  multiple  roles  at  different  times,  depending  on  team 
composition,  concept  evolution,  and  leadership  priorities.  Innovation  teams  could  have  as 
few  as  two  people  or  as  many  as  half  a  dozen  or  more,  depending  on  the  complexity  of 
the  concept.  It  would  be  the  responsibility  of  the  Chief  Innovation  Officer  to  identify 
talent  and  drive  innovation  team  composition. 


32  Mark  Rice,  project  review  meeting,  24  June  2003. 

33  Chapter  V  delves  in  detail  into  a  variety  of  analytic  tools  that  can  facilitate  a  disciplined  project  and 
portfolio  management  approach,  including  a  specific  example  of  a  potential  MDA  technology  stage- 
gate  process. 
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Phase  3  Implementation:  Supported  Concept  Managers  with  Infrastructure 

As  good  ideas  develop,  MDA  could  further  enhance  concept  development  efforts 
by  providing  some  formal  tools  and  standard  processes,  primarily  to  support  networking 
and  move  projects  toward  acquisition. 

For  instance,  a  “Network  Manager”  could  work  with  Concept  Managers  to 
develop  and  maintain  contact  and  communications  with  external  expertise,  to  encourage 
and  support  the  development  of  “communities  of  practice.”  Experts  tend  to  gravitate 
toward  individuals  with  similar  knowledge  and  interests.  Professional  networks  develop 
trust  and  awareness  between  individuals,  allowing  members  to  quickly  identify  and 
consult  with  appropriate  experts  more  efficiently  and  successfully,  all  of  which  are 
critical  to  concept  development.  A  community  of  practice  also  provides  forums  for 
ongoing  online  and  offline  interactions  between  experts  within  and  across  organizations. 
They  enable  host  entities,  from  individuals  and  academic  institutions  to  firms  and 
government  organizations,  to  efficiently  access  expertise  for  meeting  specific  objectives 
and  ongoing  talent  development. 

Active  network  development  can  greatly  facilitate  concept  development;  absent 
such  a  network,  significant  time  and  energy  would  be  required  to  identify  and  mobilize 
expertise  for  each  new  concept.  Many  industries,  such  as  telecommunications,  software, 
and  semiconductors,  combine  the  efforts  of  multiple  firms  and  organizations  into 
technology  development  consortia  to  accomplish  common  objectives,  such  as  standards 
development.  The  coordinative  and  networking  functions  of  technology  consortia  often 
require  active,  professional  management,  with  many  organized  as  separate  corporations 
with  full-time  management  teams.  Venture  capital  firms  employ  full-time  people  to 
identify  external  resources  capable  of  adding  value  to  new  ventures  and  investment 
selection.  Simply  organizing  professional  meetings  can  require  significant  effort.34 

Similarly,  a  specialist  “Process  Manager”  could  support  Concept  Managers  and 
others  in  meeting  stage-gate  approvals  and  funding  requirements.  The  Process  Manager 
could  also  supervise  a  small  information  technology  support  staff  charged  with  installing, 
populating,  maintaining,  and  mining  of  a  knowledge  base  of  MDA  S&T  programs  and 
innovation  concepts,  that  is,  an  information  system  providing  electronic  repository  of 
“organizational  memory.”  Organizational  memory  facilitates  concept  development  by 


34  Chapter  IV  discusses  various  networking  and  outreach  mechanisms  appropriate  to  the  MDA  context. 
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retaining  innovative  ideas  with  potential  to  add  value  in  the  future,  perhaps  when  a 
particular  technology  matures. 

Figure  II-3  depicts  the  evolving  structure  of  the  innovation  team.  Technology 
expertise  and  applications  expertise  interact  with  networking  and  process  support,  under 
the  direction  of  a  Concept  Manager  or  the  Chief  Innovation  Executive.  Again,  this 
depiction  is  illustrative.  Innovation  teams  would  not  have  to  follow  a  strict  structure.  A 
single  individual  could  undertake  more  than  one  role,  or  two  individuals  could  share  a 
role,  depending  on  the  specific  needs  of  the  concept  under  development.  The  form  and 
internal  functions  of  a  team  will  depend  on  the  skills  and  dispositions  of  its  members. 
Furthermore,  an  early  stage  project  might  be  refined  by  a  “core  team”  of  two  or  three 
people,  with  other  people  added  as  the  concept  evolves.  However,  clearer  team  structure 
enables  the  Chief  Innovation  Executive  to  recruit  for  particular  skills,  based  on  emerging 
areas  of  need. 


Figure  11-3.  Basic  Structure  of  an  Innovation  Team,  with  Representative  Roles 

As  MDA  innovation  activities  achieve  respect  and  acceptance,  the  group  could 
establish  more  formal  relationships  with  external  providers  of  the  types  of  variable 
resources  identified  in  Phase  1,  with  expanded  duties  as  suggested  below: 

•  Members  of  the  pool  of  technical  capabilities  could  serve  on  evaluation  boards 
and  provide  bench  scientific  support  and  due  diligence  on  the  technology 
aspects  of  new  concepts.  For  MDA,  whose  systems  are  often  specialized  and 
whose  perspective  spans  decades,  the  “pool”  may  also  serve  as  a  source  of  new 
technologies  that  underlie  new  concepts,  as  well  as  an  important  repository  of 
organizational  memory. 

•  Experts  in  scenario  building,  wargaming,  and  assessment  could  supervise 
external  concept  and  scenario  development  exercises,  in  addition  to  supporting 
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innovation  teams.  Experts  from  think  tanks  could  undertake  longer  term 
innovation  studies.35 

•  The  modeling  and  simulation  capability  could  be  expanded  and  potentially 
internalized  as  a  “center  of  excellence.” 

Also,  the  technology  stage-gate  processes  might  require  further  formalization. 
Mechanisms  must  encourage  the  timely  termination  of  unpromising  projects,  focusing 
support  on  higher  potential  projects.36  Evaluation  boards  that  implement  the  technology 
stage-gate  should  also  provide  support  for  concept  refinement.  One  option  would  be  to 
form  an  independent  expert  group,  populated  with  external  technologists,  fonner  senior 
R&D  managers,  military  planners,  and  experienced  innovation  project  managers.  This 
group  would,  where  appropriate,  be  offered  as  a  resource  to  Concept  Managers. 

Finally,  a  high-level  “Innovation  Board”  could  be  formed  to  decide  when  a 
concept  was  ready  to  become  a  formal  prototype  or  demonstration  project.  Because 
movement  through  this  gate  will  generally  require  more  substantial  budgetary  and 
technology  resources,  such  a  board  should  include  MDA  executives  and  independent 
outsiders. 

Phase  4  Implementation:  The  Application/Technology  Matrix  Organization 

The  pool  of  technical  capabilities  from  which  Concept  Managers  would  draw 
should  properly  reside  in  an  organization  that  supports  the  professional  development, 
networking,  and  refreshment  of  internal  technical  talent.  Because  MDA  does  not  have 
significant  independent  technology  identification  and  assessment  capabilities,  these 
people  will  likely  come  from  contractor,  DoD,  and  Department  of  Energy  labs.  In  the 
future,  if  MDA  S&T  investment  increases,  MDA  technology  resources  could  be 
organized  into  a  virtual  lab,  perhaps  in  the  fonn  of  different  external  labs  serving  as 
organizational  homes  for  various  strategic  technologies.  A  virtual  lab  offers  several 
mechanisms  for  addressing  the  critical  issue  of  intellectual  property  and  facilitates 
sharing.  Chapter  III  describes  the  concept  and  benefits  of  a  virtual  lab. 

Our  concern  here  is  not  with  the  details  of  how  technologists  supporting  MDA 
would  be  organized  but  rather  how  these  technologists  would  be  brought  into  regular 


35  For  example,  a  study  explaining  how  pursuing  breakthrough  technologies  and  innovative  concepts  is 
compatible  with  “spiral  development.” 

36  One  of  the  key  factors  in  constructively  ending  a  project  is  to  encourage  its  people  to  redeploy  into 
surviving  projects — not  just  by  assigning  them,  but  rather  by  finding  ways  for  them  to  buy  into  the 
new  assignment.  One  way  to  facilitate  this  process  is  to  include  the  staff  in  the  downselect  process. 
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contact  with  members  of  MDA’s  innovation  teams.  In  Phase  3  development,  interaction 
with  technologists  is  likely  to  be  mostly  ad  hoc  and  project  based.  In  Phase  4, 
“Application  Teams”  focused  on  different  aspects  of  the  ballistic  missile  defense  problem 
(e.g.,  tracking,  hard  target  kill,  etc.)  could  be  formed  to  provide  continuous  focus  on 
particular  common  issues  and  encourage  cross-fertilization  across  concept  development 
efforts.  Then,  if  some  form  of  internally  controlled  technical  pool  develops,  applications 
teams  could  be  put  in  a  matrix  with  various  strategic  technology  groups  to  make  their 
interactions  more  regular.  In  other  words,  technologists,  who  naturally  organize 
themselves  along  lines  of  technical  expertise  (e.g.,  electro-optics,  fluid  dynamics, 
engineered  materials,  etc.)  would  be  brought  in  to  serve  on  particular  Application  Teams. 
Figure  II-4  highlights  the  roles  of  application  teams  and  technology  groups. 

needs  technologies 


combination  created  by  any  team  member(s) 
candidate  projects 

Figure  11-4.  Interacting  Application  Teams  and  Core  Technology  Groups 

Foster  New  Concepts 


Prototyping  and  Demonstration 

“The  person  who  proposes  a  new  idea  is  rarely  the  best  one  to  be  put  in  charge  of 
carrying  it  out,  for  the  gifts  of  conception  and  of  execution  do  not  commonly  reside  in  the 
same  personality. ”  — Sydney  J.  Harris 

Over  time,  some  projects  will  become  ready  for  prototyping,  systems-level 
experimentation,  and  demonstration.  As  suggested  earlier,  these  projects  should  be 
managed  separately  from  concept-development  activities.  Financially,  separating 
prototyping  and  demonstration  from  initial  concept  development  isolates  these  relatively 
expensive  projects  to  avoid  diverting  resources  from  long-term  S&T  investment. 
Culturally  and  managerially,  building  operating  systems  is  a  very  different  process  from 
early  research.  Concept  development  is  visionary:  a  wide-ranging  process  aimed  at 
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discovering  new  approaches  to  problems.  Conversely,  prototyping  and  demonstration 
should  be  focused  on  closure:  refining  the  known,  making  tradeoffs,  and  implementing  as 
yet  imperfect  solutions  in  the  near  term. 

MDA/AS  could,  if  deemed  advantageous,  form  an  Advanced  Experimentation 
Office  that  focuses  on  proof-of-concept  demonstration.  Though  the  management  process 
would  differ,  selected  members  of  the  project  teams  could  transition  along  with  the 
projects,  to  convey  commitment  and  institutional  memory,  as  well  as  maintain  contact 
with  the  group  of  people  behind  the  original  concept.  Contact  with  the  original  concept- 
development  team  will  help  address  challenges  arising  during  the  prototyping  and 
experimentation  phase.37 

An  Advanced  Experimentation  Office  could  evolve  as  concepts  are  approved  for 
conceptual  and  physical  prototyping  as  a  result  of  the  stage-gate  process.  The  need  for 
mechanisms  for  prototyping  and  demonstration  will  become  evident  as  projects  mature. 
Given  their  disruptive  nature  and  resource  requirements,  resistance  may  arise  from 
existing  programs.  Prototyping  and  demonstration  will  provide  leadership  with  evidence 
to  address  such  resistance  (or  show  it  to  be  valid).  Explicit  support  from  top  MDA 
leadership  will  be  crucial  to  carry  a  program  into  true  experimentation — that  is  able  to 
sustain  setbacks  without  risk  of  immediate  cancellation — especially  when  this 
experimentation  is  costly.38 

If  an  Advanced  Experimentation  Office  exists,  it  can  take  responsibility  for  other 
efforts  that  support  innovation  but  are  not  specific  to  particular  concepts  or  projects.  For 
example,  it  could  take  responsibility  for  developing  a  center  of  excellence  in  MDA 
modeling  and  simulation,  as  suggested  above,  to  minimize  experimentation  costs.  This 
center  of  excellence  would  be  a  mechanism  through  which  the  Advanced 


37  Continuity  will  also  be  important  if,  in  the  process  of  turning  a  project  into  a  program,  difficulties  arise 
that  argue  for  returning  the  concept  to  a  previous  stage  for  further  refinement  before  undertaking  more 
proof-of-concept  experimentation. 

38  This  problem  is  not  unique  to  DoD.  Potentially  disruptive  new  products  often  fail  in  the  commercial 
arena  because  they  threaten  established,  profitable  product  lines.  At  an  early  stage  of  development,  it  is 
often  unclear  whether  the  new  product  will  be  received  favorably  by  customers,  many  of  whom  may 
be  new  customers.  (Like  DoD,  business  must  often  “create  customers”  for  innovative  new  products.) 
The  high  start-up  costs  associated  with  launching  a  new  product  mean  that  it  will  generally  be  a  near- 
term  drain  on  company  profits.  For  disruptive  products,  achieving  profitability  may  take  even  longer. 
If  they  are  forced  to  compete  against  near-term  profit  or  asset  utilization  criteria  set  by  incumbent 
business  areas,  they  may  die  before  ever  being  able  to  demonstrate  their  market  potential.  In 
enlightened  companies,  various  mechanisms  are  employed  to  foster  and  shield  innovative 
developments  from  pressure  exerted  by  their  established  product  lines.  Innovations  that  are  not 
exploited  by  the  developing  firm  often  migrate  to  competitors,  who  use  them  to  capture  market  share 
from  the  originators. 
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Experimentation  Office  obtains  early  insight  into  emerging  concepts.  It  could  also 
incubate  proven  systems  capabilities  as  they  find  operational  niches.  The  key  idea  is  to 
keep  a  new  capability  focused  on  a  limited  application  area,  iteratively  enhancing  it 
through  experimentation.  Effective  experimentation  should  enable  the  emerging 
capability  to  confront  increasingly  challenging  missions  until  it  is  sufficiently  mature, 
reliable,  and  supportable  enough  to  meet  the  demands  of  acquisition,  testing,  and 
evaluation  organizations.  Figure  II-5  below  represents  the  process.39 


Visionaries  (e.g.,  Innov.  Teams) 

Proof  of  concept 
Idea  (prototype) 


Concept 

Experimentation 


Pragmatists  (E.g.,  Services) 

Application  Procure 

Experimentation  -ment 


Development 


Figure  11-5.  Incubation  Can  Fill  the  Gap  between  Prototype  Development 

and  Initial  Application 


Transition  to  Formal  Development  Programs 

MDA/AS  does  not  control  the  acquisition  and  deployment  of  new  capabilities. 
Because  its  ideas  may  challenge  existing  programs  and  bureaucracies,  it  may  be  difficult 
to  find  eager  customers.  Many  innovation  projects  will  not  transition  to  acquisition  and 
deployment  in  a  direct  manner.  The  path  from  R&D  to  new  missile  defense  capability  is 
likely  to  be  complex  and  nonlinear,  involving  numerous  players  inside  and  outside 
government.  Long  delays  may  exist  between  proof-of-concept  and  exploitation.  As  such, 
if  fielded  disruptive  capabilities  are  the  objective,  it  will  be  insufficient  to  generate  an 
example  and  then  rely  upon  the  traditional  DoD/Service  acquisition  system  to  recognize 
its  value  and  implement  it. 

This  becomes  clear  during  the  prototyping  and  demonstration  phase.  Because  new 
capabilities  generally  lack  technical  maturity  or  operational  robustness,  the  traditional 
DoD  program-development  process  will  be  reluctant  to  undertake  the  risk-reduction 


39  Van  Atta  et  at.,  Science  and  Technology. 
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efforts  required  to  move  them  into  acquisition.40  Rapid  acquisition  and  deployment  of 
disruptive  capabilities  require  an  integrated  senior  leadership  effort,  from  the  Director  of 
MDA  and  Under  Secretary  for  Acquisition,  Technology  and  Logistics.  OSD  leaders  must 
also  have  allies  among  top  Service  leadership  to  alter  the  course  of  ordinary 
organizational  politics.  These  individuals  must  exercise  authority  to  overcome 
organizational  resistance  to  new  ideas,  uncertainty,  and  perceived  competition.41 
Whenever  possible,  transition  to  formal  development  programs  should  be  accomplished 
while  those  who  brokered  the  initial  deal  are  still  in  power. 

RECOMMENDATIONS 

In  addition  to  refinement  of  the  options  presented  herein,  the  primary  issues  to  be 
resolved  relate  to  MDA’s  specific  organizational  priorities,  capabilities,  and  readiness  to 
implement  the  recommendations  below.  The  first  few  measures  would  be  implemented 
immediately,  while  the  others  could  evolve  over  time  as  successful  early  stage  projects 
emerge. 

•  Appointment  of  a  Chief  Innovation  Officer — This  person  would  lead  and 
support  the  overall  effort,  overseeing  Concept  Managers  and  providing  funds 
to  contract  for  support  in  mission  needs  discovery  and  definition,  technology 
identification  and  assessment,  networking  and  outreach,  and  project  and 
process  management.  In  the  near  term,  we  estimate  that  funding  on  the  order  of 
$10  million  would  be  sufficient  to  begin  supporting  innovation  processes  in 
MDA. 

•  Recruitment  of  Concept  Managers — These  managers  would  create,  refine  and 
develop  innovative  concepts.  Concept  Managers  could  be  recruited  from 
among  the  extensive  group  of  people  with  missile  defense  or  related 
experience.  MDA  could  successfully  begin  its  innovation  promotion  activities 
with  about  six  or  so  Concept  Managers. 

•  Formation  of  innovation  teams — These  teams  would  be  formed  through  both 
internal  and  outsourced  support  for  Concept  Managers.  These  tenns  could  be 
made  up  of  technology  scouts  who  are  experts  in  rapid  technology  search  and 
early  concept  conception;  mission  analysts  who  have  talent  for  seeing  through 
immediate  problems  to  broader  needs  and  are  capable  of  recognizing  and 
technically  validating  promising  ideas;  and  operational  experts  who  understand 
current  applications  and  customer  cultures.  This  enhanced  organization  could 


40  R.  Van  Atta  and  M.  Lippitz,  Transition  and  Transformation:  DARPA’s  Role  in  Fostering  the 
Emerging  Revolution  in  Military  Affairs, Volume  I — Overall  Assessment,  IDA  Paper  P-3698, 
(Alexandria,  Va.:  Institute  for  Defense  Analyses,  April  2003). 

41  Ibid. 
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consist  of  something  like  20  staff  and  a  budget  on  the  order  of  $20  million, 
though  the  staff  could  be  smaller  and  the  budget  larger  depending  on  the  extent 
of  support  activity  outsourcing.  Moreover,  it  will  be  important  to  distribute 
early  funding  authority  to  prevent  a  single  individual  from  killing  an  idea  in  its 
early  stages. 

•  Implementation  of  formal  tools  and  standard  processes — These  processes 
could  include  a  “technology  stage  gate”  process  for  winnowing  out  S&T 
candidates  through  increasingly  rigorous  review;  an  independent  expert  group; 
a  high-level  “Innovation  Board”  to  decide  when  a  concept  is  ready  to  become  a 
formal  prototype  or  demonstration  project;  a  “Network  Manager”  to  develop 
and  support  “communities  of  practice”;  a  “Process  Manager”  to  manage  DoD 
bureaucratic  and  budget  requirements;  a  small  infonnation  technology  support 
staff  charged  with  installing,  populating,  maintaining,  upgrading,  and  mining  a 
knowledge  base  of  MDA  S&T  programs  and  innovation  concepts. 

•  Creation  of  mechanisms  for  regular  interaction  with  technologists — 
“Application  Teams”  could  be  fonned  to  provide  continuous  focus  on  certain 
common  missile  defense  issues  and  encourage  cross-fertilization  across 
concept-development  efforts.  Such  teams  could  help  focus  technical  experts 
from  various  disciplines  toward  ongoing  concept  development  in  terms  of 
mission  needs  and  application  requirements,  rather  than  in  tenns  of  technology 
specializations. 

•  Formation  of  an  Advanced  Experimentation  Office — This  office  would  focus 
on  proof-of-concept  experimentation  and  prototype  demonstration.  Early  on, 
this  would  take  the  form  of  a  planning  function  that  anticipates  implementation 
issues  of  maturing  concepts.  When  projects  are  ready  for  prototyping  and 
demonstration,  they  should  be  managed  separately  from  concept-development 
activities  because  of  the  differing  nature  of  the  work  and  higher  costs. 
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III.  STRATEGIC  TECHNOLOGY  STEERING  &  EXECUTION 


Lee  Kindberg  and  Marius  Vassiliou 


OVERVIEW 

Strategic  technology  steering  refers  to  mechanisms  useful  for  high-level 
management  of  a  portfolio  of  long-term  S&T  projects  in  a  limited  number  of  technology 
areas  deemed  critical  to  the  long-term  success  of  the  MDA  mission.  This  chapter  outlines 
the  strategic  technology  steering  concept  and  suggests  ways  MDA  could  implement 
elements  in  the  near  term,  without  significant  organizational  change,  while  laying  a  basis 
for  an  expanded  effort  linked  to  a  long-range  S&T  steering  and  execution  strategy.  It 
concludes  with  a  discussion  of  the  use  of  virtual  laboratory  mechanisms  for  executing 
S&T  in  strategic  technology  areas,  including  a  way  of  involving  S&T  contractors 
consistent  with  MDA  organizational  capabilities,  needs,  and  outlook,  as  well  as  legal  and 
regulatory  constraints. 

THE  ORGANIZATIONAL  TECHNOLOGY  STEERING  PROBLEM 

Technology  steering  is  intended  to  focus  resources  and  management  attention  on 
those  critical  few  technologies  that  have  long-tenn  strategic  impact  on  the  success  of  the 
organization’s  overall  mission.1  The  intent  is  to  maximize  S&T  productivity, 
effectiveness,  and  timeliness  in  these  technologies — often  designated  as  “key,”  “core,”  or 
“strategic”  technologies,  the  terminology  we  have  adopted  here.  Effective  technology 
steering  mechanisms  must  address  a  number  of  organizational  needs,  both  from  a 
management  perspective  and  at  the  technologist  level,  including  resource  allocation 
mechanisms,  management  tools,  internal  and  external  technical  communications  and 
collaborations,  technology  scouting  systems,  and  development  and  stewardship  of 
technical  personnel.  Ideally,  technology  steering  is  a  component  of  an  integrated  S&T 
strategy  that  also  includes  idea  generation  and  transition  to  operating  divisions  for  use  or 
deployment. 


1  “Critical  few”  refers  to  the  quality  management  concept  described  by  Dr.  W.  E.  Deming  and  others 
that  organizations  must  identify  and  focus  on  the  critical  few  issues  or  problems  most  likely  to  have 
significant  impact,  distinguishing  them  from  the  “trivial  many”  problems  facing  any  manager. 
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Technology  Steering  approaches  should  be  designed  to  be  long-tenn  in  vision  and 
time  frame,  yet  sufficiently  flexible  to  meet  changing  organizational  needs,  mission 
targets,  and  management  priorities.  These  approaches  must  be  structured  to  weather 
organizational  restructurings  and  changing  management  personalities,  and  still  adapt  to 
meet  the  organization’s  needs  in  that  time  frame  and  environment;  at  times,  these 
approaches  must  respond  to  address  immediate  challenges.  These  approaches  and  tools 
often  lead  to  the  development  or  expansion  of  new  technical  competencies  in  the 
organization. 

THE  MDA  IMPLEMENTATION  ENVIRONMENT 

The  missile  defense  mission  has  evolved  over  the  years  and  taken  on  new  aspects 
in  the  most  recent  reorganization  to  fonn  MDA.  While  technology  has  been  critical  to 
mission  success  throughout  the  organization’s  existence,  organizational  structure  and 
resources  committed  to  long-term  S&T  have  varied.  This  has  likely  contributed  to 
personnel  turnover  in  long-term  technical  roles  and  a  possible  loss  of  organizational 
memory.  With  leadership  movement  between  MDA  and  other  DoD  organizations  and 
with  military  personnel  promotion  systems  that  create  turnover,  leadership  tenure  has 
also  tended  to  be  short  when  compared  to  the  private  sector.  At  present,  MDA  is  focused 
on  implementation,  with  longer  term  S&T  de-emphasized.  No  formal  long-range  MDA 
Strategic  Technology  Steering  function  currently  exists,  though  a  recent  draft  Integrated 
Program  Plan  identifies  the  need  for  “frequent  close  coordination  between  MDA/AS  and 
Element  Program  Directors... to  ensure  all  technology  areas  identified  in  the  Technology 
Roadmap  are  pursued. . . .”  (See  Figure  1-2) 

Adapting  commercial  models  to  MDA  will  often  not  be  straightforward  because 
of  MDA’s  contract  research  environment  and  federal  regulations  governing  contractor 
participation  on  advisory  bodies.  In  addition,  selecting  the  right  initial  leader  and  team 
will  be  critical  to  successful  implementation.  MDA  may  not  currently  have  the  ideal  mix 
of  personnel  to  initiate  implementation  of  a  vigorous  long-tenn  S&T  program,  or  if 
present,  they  may  be  assigned  to  more  immediate  programs.  MDA’s  history  of 
organizational  change  may  make  it  difficult  to  attract  and  retain  the  long-range  S&T 
personnel  needed,  so  time  and  creative  approaches  may  be  required  to  research  staffing. 
The  discussion  of  “virtual  laboratory”  possibilities  at  the  end  of  this  chapter  provides 
possible  alternative  structures  aimed  at  overcoming  these  difficulties. 
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IMPLEMENTATION  OPTIONS 


A  range  of  options  and  variations  are  available  to  MDA  to  implement  technology 
steering  mechanisms.  Technology  steering  functions  in  the  public-  and  private-sector 
cases  studied  in  Phase  I  ranged  from  purely  technical  networks  to  full  technology- 
management  systems  covering  the  total  S&T  effort.  Some  managed  people  and  resources, 
selected  technical  directions,  and  articulated  the  long-tenn  strategy  of  the  organization. 
Others  focused  just  on  specific  technologies.  All  dealt  with  tracking  or  driving 
innovations  in  targeted  areas,  often  in  partnership  with  key  contractors  or  partners.  The 
following  technology  steering  models  were  deemed  to  have  strong  potential  as  options 
for  MDA. 

Key  Technology  Steering  Groups 

A  well-defined  example  of  a  technology  steering  process  was  put  in  place  at  ITT 
in  the  1970s  by  Dr.  Charles  Herzfeld,  a  fonner  DARPA  director.  ITT’s  Key  Technology 
Steering  Groups  were  established  to  develop  and  manage  S&T  programs  in  selected 
priority  technology  areas.  Key  Technology  Steering  Groups  determined  needs,  set 
objectives,  and  tracked  technical  progress,  with  the  intent  of  focusing  and  linking  the 
programs  to  product  needs,  but  they  were  not  involved  in  the  specific  day-to-day  conduct 
of  individual  projects  within  the  thrust.  Establishing  Key  Technology  Steering  Groups 
was  a  function  of  high-level  technology  management,  as  part  of  the  overall  technology 
management  process.  Key  Technology  Steering  Groups  reported  to  the  Chief  Scientist 
(the  highest  level  corporate  technical  management),  who  had  an  active  role  in  research 
management  and  regular  access  to  the  Chief  Executive  Officer.  Membership  included  the 
best  domain  experts  for  the  technology  subject,  serious  potential  users  of  the  technology 
to  be  developed,  and  the  main  producers  of  the  technology,  with  outside  experts  as  guests 
and  advisors.  The  chairs  were  chosen  from  the  experts  in  the  field. 

“Next  generation  technologies”  group  established  in  an  existing  S&T  organization 

The  DuPont  APEX  Research  Model  is  a  fonnal  process  for  portfolio  and  project 
management  as  part  of  DuPont’s  overall  long-term  central  R&D  effort.  An  APEX 
Science  Board,  led  by  the  Chief  Science  &  Technology  Officer,  oversees  Science  Boards 
associated  with  each  of  the  company’s  three  “strategic  platforms.”  These  boards  include 
the  highest-level  technical  leadership  and  require  business  unit  involvement  even  at  the 
earliest  stages  of  research.  A  highly  structured  three-stage  “Technology  Stage  Gate” 
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process  is  used  to  evaluate  projects  for  resources  and  inclusion  in  the  next  stage  of  the 
development  portfolio. 

Formal  collaborative  models 

Integrated  High  Performance  Turbine  Engine  Technology  Program 

The  Integrated  High  Perfonnance  Turbine  Engine  Technology  Program 
(IHPTET)  started  in  the  1980s  (and  is  now  transitioning  to  VAATE,  the  Versatile 
Affordable  Advanced  Turbine  Engines  Program).  IHPTET  aimed  to  bring  coherence  and 
encourage  collaboration  in  turbine  engine  R&D  perfonned  by  contractors  sponsored  by 
the  U.S.  Air  Force.  The  IHPTET  Steering  Committee  is  led  by  an  OSD  S&T  executive, 
with  membership  including  all  the  user  communities  affected  by  turbine  engine 
technology  [the  Services,  National  Aeronautics  and  Space  Administration  (NASA), 
Department  of  Energy].  Linking  research  with  applications  encourages  technology 
transition  and  builds  user  community  support  for  the  long-tenn  research  program  and 
budget.  The  Steering  Committee  oversees  an  Industry  Panel  and  three  types  of 
technology  panels:  Technology  Focus  Area  Panels,  Pervasive  Technology  Panels,  and 
Technology  Application  Panels.  The  focus  area  panels  and  specific  technology  panels 
function  like  the  Key  Technology  Steering  Groups  in  the  ITT  model.  Panel  members, 
who  represent  the  teams  that  actually  do  the  research,  are  generally  not  members  of  the 
Steering  Committee.  Because  turbine  engine  technology  is  mature,  IHPTET  panels  focus 
on  measurable  goals  and  objectives  as  opposed  to  new  concepts,  radical  innovations,  and 
quantum  improvements  in  performance. 

National  Technology  Alliance 

The  National  Technology  Alliance  is  a  program  run  by  National  Geospatial 
Intelligence  Agency  to  discover,  initiate,  or  accelerate  development  and  exploitation  of 
commercially  available  solutions  to  meet  technology  needs  for  intelligence  applications. 
The  National  Technology  Alliance  (1)  acquires  knowledge  of  Government  operational 
user  needs  by  conducting  technology  needs  assessment  and  analysis;  (2)  seeks  out 
commercial  technology  solutions;  (3)  performs  R&D  aimed  at  leveraging  commercially 
developed  technology;  and  (4)  provides  a  forum  for  exchange  of  Government  technology 
needs,  solutions,  and  experimentation.  The  National  Technology  Alliance  funds  dual-use 
R&D  projects  (1-5  years  out)  and  prototype  development  in  selected  technical  areas. 
Technical  focuses  are  determined  through  a  review  of  published  DoD  needs  statements,  a 
proprietary  review  process,  and  input  from  DoD  laboratory  representatives.  Respondents 
are  consortia  of  companies  and  academic  centers  that  respond  to  a  Broad  Area 
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Announcement.  Task  orders  are  then  issued  to  consortia  to  develop  specific  products,  in 
coordination  with  agencies  and  end  users.  A  private  organization,  the  Rosettex 
Technology  &  Ventures  Group,  was  created  to  advance  and  commercialize  technologies 
of  high  interest  to  The  National  Technology  Alliance.  Rosettex  has  a  team  of  more  than 
60  partners  representing  major  technology  consulting  firms,  established  and  new 
companies,  independent  research  institutes,  academic  institutes,  and  government 
contractors.  Members  of  the  consortium  were  chosen  so  that  Rosettex  could  address  the 
entire  time  line  of  technology  development  up  to  acquisition.  They  are  divided  into 
groups  organized  around  stages  on  that  time  line  to  eliminate  conflicts  of  interest  and  so 
to  foster  free  exchange  of  technical  infonnation  at  the  precompetitive  level  while 
protecting  intellectual  property  rights. 

Network  of  interested  professionals  with  Network  Executive  Committee  (Hughes) 

Hughes  fonned  voluntary  networks  of  technical  professionals  in  identified  critical 
technology  areas,  each  of  which  was  led  by  a  Network  Executive  Committee  (essentially 
a  Key  Technology  Steering  Group  of  6-12  representatives  from  each  business  group 
involved).  Each  representative  was  a  person  with  a  technical  background  and  good 
experience  in  the  subject  technology,  who  also  controled  resources.  Typical  members 
might  be  laboratory  managers  (heading  engineering  organizations  of  100-300  people)  or 
chief  scientists  or  engineers.  The  “control  of  resources”  qualification  might  mean  line 
control,  program  control,  or  simply  control  through  influence  and  respect.  This  approach 
has  the  advantages  of  addressing  both  the  management  and  steering  needs  (top  down)  and 
the  technologists’  needs  for  interaction  and  communications  (bottom  up). 

A  STRATEGIC  TECHNOLOGY  STEERING  GROUP  FOR  MDA 

A  Strategic  Technology  Steering  Group  should  be  long-range  and  strategic  in 
view,  structure,  and  membership.  These  functions  are  intended  to  “guide”  new 
exploratory  research  in  areas  of  particular  importance  to  the  organization.  Thus,  for 
MDA,  Strategic  Technology  Steering  Groups  would  focus  on  technologies  critical  to 
long-term  systems  and  future-generation  strategic  needs,  beyond  currently  planned 
blocks  upgrades  and  spanning  multiple  blocks.  To  be  most  effective,  technology  steering 
leadership  should  have  knowledge  of  the  strategic  technology  and  its  mission 
implications,  ties  to  the  leading  practitioners  of  that  technology,  and  authority  to  commit 
resources.  Common  characteristics  of  successful  Strategic  Technology  Steering  Groups 
include  the  following: 
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•  High-level  support,  visibility  and  engagement. 

•  The  groups  offer  long-term  technical  leadership  and  active  oversight  for  a 
particular  S&T  thrust  of  particular  importance  to  the  organization,  but  they  are 
not  involved  in  day-to-day  management  of  individual  projects. 

•  Chairs  of  particular  Strategic  Technology  Steering  Groups  are  acknowledged 
experts  in  a  technical  community  directly  relevant  to  the  thrust  area. 

•  Membership  emphasizes  expertise  and  competence,  not  position,  and  includes 
the  best  domain  experts  for  the  technology  and  serious  potential  users  of  the 
technology  to  be  developed.  Membership  may  include  the  main  producers  of 
the  technology,  and  it  should  include  outside  experts  as  guests  and  advisors. 

•  Steering  group  meetings  meet  regularly — initially  once  per  month,  moving  to 
2-3  month  intervals.  The  meetings  are  at  places  where  the  S&T  work  is  done 
(research  facility  or  contractor’s  location).  The  sessions  should  be  closed  and 
discussions  classified  or  confidential. 

•  They  must  act  as  and  become  teams,  not  just  committees. 

•  They  must  be  structured  to  weather  short- tenn  organizational  changes. 

Organizations  often  transition  into  a  Strategic  Technology  Steering  Group 
structure  by  starting  with  a  small  pilot  or  prototype  to  demonstrate  and  fine  tune  the 
concept,  which  then  is  applied  to  additional  technology  areas.  Successful  concepts  are 
often  adopted  by  other  parts  of  the  organization.  Table  III- 1  gives  a  menu  of  components 
from  which  MDA  can  build  an  initial  prototype  and  then  expand  it  as  organizational 
needs  are  defined  or  the  environment  becomes  more  receptive.  While  some  clarification 
of  the  menu  options  is  required,  the  primary  issues  to  be  resolved  relate  to  MDA’s 
specific  organizational  priorities,  capability,  and  readiness  to  implement  these  measures. 
It  will  then  be  possible  to  select  a  target  technology  for  the  prototype  Strategic 
Technology  Steering  Group  and  further  define  the  necessary  structures. 
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Table  MI-1.  Prototype  Components  and  Options 


Organizational 

Need 

Strategic  Technology  Steering  Group  Prototype  Components  and 
Implementation  Steps 

Technology  Steering 

1 .  Identify  technology  for  first  implementation.  (Link  to  MDA  Technology  Roadmap  and 

strategic  planning  processes.) 

2.  Establish  Strategic  Technology  Steering  Group  focused  on  that  technology. 

•  Respected  leader  assigned  at  the  25-50%  effort  level. 

•  Members  from  MDA/AS  and  other  MDA  elements  for  which  this  technology  is  critical 
(essential  to  success) 

•  Identified  top  management  sponsor/champion,  who  must  be  committed,  engaged,  and 
kept  informed. 

•  Connection  to  technology  resources  outside  MDA  (options  below). 

•  Responsible  for  tracking  and  reporting  on  technology,  prioritizing  efforts,  identifying 
unfilled  needs,  evaluating  new  proposals,  and  output  of  “Innovation  Hub.” 

•  Provide  substantial  input  to  future  Technology  Roadmaps  and  strategic  plans. 

3.  Support  with  voluntary  networking  of  related  technology  community  (see  below). 

4.  With  growth,  expand  structure  both  up  and  down,  following  the  IHPTET  model: 

•  Add  additional  Strategic  Technology  Steering  Groups  as  critical  technologies  are 
identified. 

•  With  multiple  Strategic  Technology  Steering  Groups  in  place,  implement  an  MDA 
Technology  Council  as  (1)  an  oversight  board  to  prioritize  and  direct  the  long-term 
efforts  and  as  (2)  stewards  of  the  Technology  Roadmap  and  these  critical  long-range 
technology  thrusts. 

High  level  support 

1 .  Executive  sponsor  or  direct  reporting  relationship  for  Strategic  Technology  Steering 

Group. 

2.  Strategic  Technology  Steering  Group  participation  by  key  element  program  leaders. 

3.  Commitment  to  provide  strong  Strategic  Technology  Steering  Group  leader(s)  and 
member(s). 

4.  Clear  definition  of  expectations  for  communication  to  upper  management. 

(These  areas  are  critical  for  success.) 

Coordinate  efforts 
across  MDA  (to 
communicate  needs 
and  develop  support) 

A  Hughes-style  Strategic  Technology  Network  would  enhance  the  capability  of  the  Strategic 
Technology  Steering  Group  to  identify  and  meet  needs,  prevent  duplication  of  effort,  and 
encourage  collaboration.  (Networking  is  addressed  further  in  Chapter  IV.) 

Engage  user 
community 

The  IHPTET  Steering  Committee  included  representatives  from  all  key  DoD  stakeholders. 
Depending  on  the  technology(ies)  selected,  the  structure  established  should  accommodate 
inclusion  of  both  internal  and  other  DoD  experts  and  users. 

A  planned  series  of  long-range  S&T  symposiums  (limited  attendance?)  would  showcase  and 
communicate  new  concepts  and  needs,  provide  recognition  for  strong  contributors  and 
innovators,  and  increase  interaction  across  the  organization  and  support  community. 

Track  and  engage 
outside  technology 
expertise 

1 .  The  IHPTET  Industry  Panel  model  would  allow  participation  by  contractor  personnel  with 
critical  knowledge  while  complying  with  legal  requirements  on  contractors  serving  on 
advisory  bodies.  This  participation  could  vary  in  roles,  levels,  and  duration  and  include 
participants  who  are  not  long-term  “members”  of  the  team. 

2.  If  interactions  are  to  extend  to  R&D  organizations  beyond  the  typical  DoD  contractors,  a 
CBRTA-type  coalition  contracting  structure  would  allow  pre-competitive  interaction  of 
companies  on  critical  R&D  while  protecting  intellectual  property  rights  and  expediting 
contracting.  This  could  be  established  most  expeditiously  as  an  additional  “BIN”  under  the 
National  Technology  Alliance  umbrella. 

Tools  for  advancing 
projects  and  allocating 
resources 

Implement  a  modified  technology  stage-gate  process  similar  to  the  DuPont  APEX  structure, 
with  modifications  to  address  MDA’s  mission  specifics,  longer  time  horizon,  and  contract 
environment. 

VIRTUAL  LABORATORIES  FOR  EXECUTION  OF  STRATEGIC  S&T 

As  the  strategic  technology  steering  framework  comes  into  being,  MDA  could 
benefit  from  developing  more  formal  structures  for  executing  S&T  programs  in  strategic 
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technology  areas.  This  section  describes  how  MDA  could  create  a  close-knit 
collaborative  structure  with  expert  organizations  in  various  areas  to  create  a  virtual 
laboratory.2 

What  is  a  Virtual  Laboratory? 

In  recent  years,  the  outsourcing  of  research  has  been  occurring  at  an  increasing 
pace  in  a  wide  variety  of  industries,  both  to  decrease  costs  and  to  gain  access  to  unique 
capabilities  present  elsewhere.3  A  virtual  laboratory  is  a  highly  collaborative  form  of 
outsourcing,  with  significant  integration  of  the  various  organizations  involved  and  joint 
management  processes  of  considerable  breadth  and  authority.  The  involvement  of  the 
central  organization  in  the  execution  and  management  of  the  research,  and  the  depth  of 
commitment  of  the  various  partners,  is  such  that  the  collaboration  looks  almost  as  if 
everyone  belonged  to  the  same  research  organization. 

A  virtual  laboratory  occupies  the  middle  ground  between  perfonning  research  in- 
house  on  the  one  hand  and  simply  issuing  research  contracts  to  external  organizations  on 
the  other  (see  Figure  III- 1).  Consider  first  the  right  side  of  Figure  III- 1 .  To  the  far  right 
are  in-house  research  organizations,  of  which  there  have  been  a  number  throughout  the 
last  40  years.  Xerox’s  Palo  Alto  Research  Center,  Hewlett-Packard  Laboratories,  and 
IBM  Thomas  J.  Watson  Research  Center  are  among  many  examples,  as  are  some 
National  Laboratories.  Moving  left  in  the  figure,  we  encounter  more  complex  R&D 
organizations,  with  geographically  distributed  laboratories  (nationally  or  internationally). 
The  global  research  organizations  of  Sony  and  Matsushita  are  examples  of  these,  as  is 
Boeing’s  Phantom  Works.4  Notwithstanding  the  increased  complexity,  these  are  not 
virtual  laboratories  by  our  definition.  They  are  geographically  distributed  research 
organizations  affiliated  with  the  same  parent  organization. 


2  M.  G.  Russell,  “The  ‘Virtual  Laboratory’:  Alliances  for  Technology  Transfer,”  Proc.  Twenty-Seventh 
Annual  Hawaii  International  Conference  on  System  Sciences,  1994,  pp.  478-482. 

3  S.  J.  Kobrin,  J.  T.  Battenberg,  P.  Hewitt,  P.  J.  Jennings,  J.  Joerres,  S.  Kumar,  F.  Mer, 
“Worldsourcing’s  Next  Frontier:  R&D,”  World  Economic  Foram  Annual  Meeting,  Davos, 
Switzerland,  2004;  C.  H.  Kimzey  and  S.  Kurokawa  (2002),  “Technology  Outsourcing  in  the  U.  S.  and 
Japan,”  Research  and  Technology’  Management,  Jul.-Aug.  2002,  pp.  36M2;  R.  Varma,  “Changing 
Research  Cultures  in  U.S.  Industry,”  Science,  Technology’,  and  Human  Values  25,  2000,  pp.  395-416; 
A.  H.  Rubenstein,  “Coping  with  Downsizing  and  Outsourcing:  The  Virtual  Corporate  Research  Lab,” 
Proc.  Portland  International  Conference  on  Technology’  and  Innovation  Management  (PICMET  99)  v. 
2,  1999,  pp.  434-437. 

4  S.  Arimura,  “How  Matsushita  Electric  and  Sony  Manage  Global  R&D,”  Research  and  Technology’ 
Management,  Mar-Apr  1999,  pp.  41-52;  J.  Fricker,  “Boeing’s  Phantom  Works:  Shaping  the  Future,” 
www.aviationnow.com,  18  June  2001. 
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Figure  MI-1.  The  Virtual  Laboratory  Continuum 

Consider  now  the  left  side  of  Figure  III- 1 .  To  the  far  left  are  contracting 
organizations  such  as  the  National  Science  Foundation,  the  Office  of  Naval  Research, 
and  DARPA.  Despite  the  intelligence  and  creativity  of  the  contract  monitors  and  program 
managers  at  such  institutions,  their  mission  is  not  to  perform  original  research  internally, 
but  rather  to  let  contracts  to  external  organizations.  These  external  organizations  execute 
the  contracts  with  varying  levels  of  guidance  and  report  their  results.  In  some  cases,  the 
involvement  of  the  contracting  organization  increases.  This  entails  a  move  away  from  the 
left  end  of  the  figure  and  toward  the  center.  There  are  collaborations  among  government 
agencies  for  large  and  complex  contract  research  programs,  which  can  be  thought  of  as 
“virtual  agencies.”5  Examples  of  large  programs  managed  by  virtual  agencies  include  the 
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M.  Castro,  R.  S.  Foster,  K.  Gunn,  and  E.  R.  Roberts,  “Managing  R&D  Alliances  Within  Government: 
The  ‘Virtual  Agency’  Concept,”  IEEE  Transactions  on  Engineering  Management  50,  2003,  pp.  297- 
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Partnership  for  a  New  Generation  Vehicle,6  the  High  Performance  Computing  and 
Communications  program,  and  the  Next-Generation  Internet.  These  virtual  agencies 
engage  in  significant  and  activist  collaborative  planning,  but  they  are  merely  complex 
contracting  organizations,  not  virtual  laboratories.  Only  as  a  contracting  organization 
becomes  more  involved  in  the  actual  execution  of  the  contracts  it  lets  does  the  assembled 
enterprise  take  on  more  of  the  character  of  a  virtual  laboratory. 

In  the  center  of  the  figure  is  the  virtual  laboratory  space,  designated  by  the  dashed 
oval.  The  coordination  and  commitment  of  the  partnerships  here  are  at  a  maximum,  as  is 
the  incremental  management  overhead  associated  with  the  collaborations.  Some  virtual 
laboratories  are  formed  when  research  organizations  undertake  significant  outsourcing  to 
partners,  with  whom  there  is  very  close  coordination.  These  partnerships  often  involve 
personnel  representing  the  contracting  organization  spending  significant  amounts  of  time 
and  energy  working  with  partner  organization  personnel.  For  example,  Pacitti  and 
Pearson  describe  the  case  of  a  large  electronics  company  which  they  call  Company  B. 
This  company,  after  severely  downsizing  its  central  R&D  laboratory,  rebuilt  its  research 
infrastructure  in  a  completely  distributed  fashion,  with  small  local  laboratories  all  over 
the  world.  Some  of  these  laboratories  are  internal  to  the  company,  and  some  are  external. 
Here,  a  significant  amount  of  central  direction  and  planning  is  balanced  with  local 
autonomy.  The  entire  assemblage  appears  to  constitute  an  effectively  functioning  virtual 
laboratory.7  Another  example  is  the  relationship  of  Martin  Marietta  laboratories  with 
General  Electric  in  the  wake  of  the  sale  of  GE’s  aerospace  business  to  Martin  Marietta 
(later  part  of  Lockheed  Martin).8  A  similar  relationship  is  that  of  Rockwell  Scientific 
with  Boeing  Phantom  Works  after  the  sale  of  Rockwell’s  aerospace  divisions  to  Boeing. 
Rockwell  Scientific  has  functioned  as  a  virtual  division  of  Phantom  Works  for  the  last  8 
years.  (This  case  is  discussed  in  more  detail  in  Appendix  A.) 

The  Army  Research  Laboratory’s  (ARL)  Advanced  Displays  Federated 
Laboratory  is  particularly  relevant  to  MDA  because  it  was  led  by  a  government  research 
laboratory.  The  Advanced  Displays  Federated  Laboratory  comprised  a  tightly  knit 
collaborating  consortium  of  industrial  and  academic  research  institutions  led  by  ARL  in 


6  National  Academy  of  Sciences,  Review  of  the  Research  Program  of  the  Partnership  for  a  New 
Generation  of  Vehicles:  Seventh  Report ,  National  Academy  Press,  2001. 

7  B.  Pacitti,  and  A.  Pearson,  “Organizational  Networks  in  R&D,”  Proc.  International  Conference  on 
Engineering  and  Technology’  Management  (IEMC  96),  1996,  pp.  428-433. 

8  United  States  General  Accounting  Office,  Defense  Restructuring  Costs:  Projected  and  Actual  Savings 
from  Martin  Marietta  Acquisition  of  GE  Aerospace,  Letter  Report,  09/05/96,  GAO/NSIAD-96-191. 
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the  pursuit  of  common  goals.  The  ARL  Advanced  Displays  Federated  Laboratory  is  also 
discussed  in  detail  in  Appendix  A. 

Implications  for  MDA 

A  virtual  laboratory  structure  may  provide  MDA  with  a  viable  method  of 
accessing  talent  and  capabilities  that  would  be  very  difficult  or  impossible  to  assemble 
and  retain  at  any  cost.  A  virtual  laboratory  would  also  allow  for  arrangements  that  are 
more  flexible  and  with  fewer  human  resource  constraints  than  if  all  capabilities  were 
maintained  in-house,  while  allowing  for  better  understanding,  control  and  coordination 
than  simple  contracting.  On  the  other  hand,  a  virtual  laboratory  involves  higher 
management  overhead,  less  flexibility,  and  higher  cost  than  pure  contracting,  at  least  in 
the  near  tenn.  Conflicts  of  interest  between  the  institutions  making  up  the  virtual 
laboratory  divisions  must  also  be  managed.  Experience  suggests  that  a  number  of  factors 
associated  with  successful  virtual  laboratories  could  be  implemented  for  MDA: 

•  Consistent,  stable,  multiyear  funding  for  the  enterprise.  There  must  be  an 
incentive  for  external  organizations  to  invest  the  time  and  effort  to  become 
virtual  divisions. 

•  A  dedicated  Virtual  Lab  manager  at  the  contracting  agency  (likely  MDA/AS) 
and  a  dedicated  leader  at  the  designated  prime  external  organization.  Both 
individuals  must  be  energetic  champions  and  must  have  real  and  widely 
recognized  authority. 

•  Dedicated  local  managers  and  staff  at  each  virtual  division. 

•  Highly  knowledgeable  active  researchers  at  MDA/AS  to  engage  with  virtual 
divisions. 

•  Some  coordinated  control  of  incentives  (personnel  evaluations  and 
compensation). 

•  Unambiguous  agreement  among  virtual  divisions  to  let  work  be  centrally 
directed. 

•  Collaborative  planning  with  significant  input  from  virtual  divisions. 

•  Arbitration  mechanisms  to  handle  conflicts  of  interest. 

•  Staff  rotation  in  and  out  of  MDA/AS  and  between  virtual  divisions. 
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CONCLUSION 


This  chapter  has  outlined  principles  and  methods  for  management  and  execution 
of  strategic  missile  defense  technologies.  Specific  implementation  steps  over  time  could 
include  the  following: 

•  Establishing  a  Strategic  Technology  Steering  Group  focused  on  a  selected 
technology  with  a  respected  leader  and,  critically,  direct  reporting  to 
supportive,  committed  top  management. 

•  Supporting  the  Strategic  Technology  Steering  Group  with  networking  of 
related  technology  communities,  potentially  including  industry  panels  created 
to  allow  contractor  participation. 

•  Implementing  a  technology  stage-gate  process  with  modifications  to  address 
MDA’s  mission  specifics,  longer  time  horizon,  and  contract  environment. 

•  Developing  virtual  laboratories  to  perform  long-term  S&T,  that  is,  a  highly 
collaborative  fonn  of  S&T  outsourcing  employing  formal,  precompetitive 
coalition  contracting  structures  to  permit  interaction  of  companies  on  S&T 
while  protecting  intellectual  property  rights. 
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IV.  NETWORKING  AND  OUTREACH  FOR 
MDA  SCIENCE  AND  TECHNOLOGY 


Daniel  Petonito 


INTRODUCTION 

This  chapter  describes  a  phased  implementation  plan  for  best  practices  in  internal 
and  external  networking  and  outreach.  Networking  and  outreach  are  different  functions 
with  different  objectives.  Networking  is  primarily  about  processes  used  to  interact  with 
the  scientific  community  to  identify  promising  technologies,  potential  emerging  threats, 
gaps  in  current  research,  and  partnering  opportunities.  It  has  an  external  aspect,  aimed  at 
obtaining  independent  evaluations,  and  an  internal  aspect,  aimed  at  improving 
coordination  on  the  MDA  research  agenda.  Outreach  has  more  of  a  customer  relations 
aspect.  While  it  can  be  directed  to  the  scientific  community,  it  will  more  often  be  directed 
toward  the  leadership  of  MDA  and  its  major  programs  elements  (“the  Elements”).  The 
primary  function  of  outreach  is  to  maintain  support  for  a  healthy  long-term  research 
program  by  explaining  and  demonstrating  the  value  of  long-tenn  research  in  supporting 
the  MDA  mission. 

IMPLEMENTATION  ENVIRONMENT 

Networking  and  outreach  are  critical  parts  of  a  long-term  S&T  program.  A  review 
of  case  studies  from  previous  research  for  MDA1  identified  five  key  attributes  that 
networking  and  outreach  can  support.  These  attributes  were  present  in  organizations  with 
a  successful  long-term  S&T  program.  Table  IV- 1  shows  the  key  attributes  and  specific 
functions  identified.  Each  is  discussed  in  turn. 
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Table  IV-1.  Key  S&T  Performance  Attributes  Supported  by  Networking  and  Outreach 


Maintain  Support  for  SIT  Program 

Maintain  Top  management  Support 

Organizational  Leader  Directly  Involved  in  S&T  Decisions 

Validate  Program  to  Management  &  Outside  Stakeholders 


XXX 
X  X 
X 


X  X  X  X 
XXX 
X  X 


X  X  X  X 
X 

X  X  X  X 


X  X 
X 
X 


Connect  Research  with  Strategic  Objectives 

Create  Bold  Visions  and  Mission  Statements 
Identify  Key/Core  Technologies 

Link  S&T  to  Strategic  Objectives  -  Reassess  Periodically 

Create  Technology  Plan  or  Roadmap 

Establish  Steering  and/or  Technology  Committees 


X 

X  X 

X  X 

X  X 

X 


X  X 
XXX 
XXX 
X 
X 


X 

X 

XXX 


X  X 
X 

X  X 
X  X 
X  X 


Support  Near-term  Development 

Link  S&T  Program  Including  Project  ID  with  Divisions 
Create  Joint  Projects  with  Product  Divisions 
Provide  Technical  Problem  Solving  Capability 
Identify  Gaps,  Common  Needs  and  Exploit  Synergy 
Understand  the  Needs  of  the  User 


XXXXX  XXXXXXX 
X  XX 

X  XXXXXXXXXXX 
X  X  X  X  X  X 

XX  XXX  X 


Seek  and  Obtain  Outside  Technical  Advice  &  Technologies 

Conduct  University  Research  Program  X  X 

Seek  and  Obtain  Technology  &  Technical  Advice  X  X 

Form  Outside  Advisory  Committee  for  Independent  Review 
Keep  Abreast  of  Scientific  Achievements  around  the  World  X 


XX  X 

XX  X  X  X  X  X 
X  XX 

XX  XX 


Establish  Partnerships  with  Tech  Leaders  Worldwide 

Establish  Partnerships  with  Technology  Leaders 
Conduct  University  Research  program 


XXX  XX  X 

XX  X  X 


X 


X 


Maintain  Support  for  S&T  Program — An  attribute  shared  by  the  organizations 
studied  was  the  need  to  gain  and  maintain  support  within  the  organization  for  a  long-term 
research  program.  This  support  must  come  from  the  highest  levels  in  the  organization.  In 
industry,  this  translates  to  the  Chief  Executive  Officer  and  the  Corporate  Board,  which 
together  are  directly  involved  in  S&T  decisions  and  actively  support  a  strong  research 
program.  Support  from  the  development  groups  in  industry  or  the  Elements  in  MDA  is 
also  important  to  maintain  a  strong,  long-tenn  S&T  program.  This  is  especially  true 
during  budget  resource  allocation  decisions.  If  the  Elements  have  not  seen  or  received 
value  from  long-term  research,  their  support  for  the  program  is  unlikely.  Mechanisms  to 
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provide  strong  and  continued  support  for  a  long-tenn  S&T  program  will  have  to  be  part 
of  the  overall  implementation  strategy. 

Connect  Research  with  Strategic  Objectives — One  of  the  first  ways  to  gain 
support  for  a  long-term  S&T  program  is  to  connect  it  to  the  strategic  objectives  of  the 
organization.  Without  this  connection,  it  will  be  difficult  to  show  how  long-tenn  S&T  is 
providing  value  to  the  organization  or  to  maintain  support  for  a  healthy  program.  To 
ensure  this  connection,  oversight  and  decision-making  for  a  long-term  S&T  program  will 
have  to  come  from  top  MDA  executives. 

Support  Near-term  Development — The  primary  reason  to  support  the  Elements  is 
to  help  the  organization  achieve  its  mission.  A  primary  benefit  of  this  support  is  that  it 
demonstrates  the  value  of  a  long-term  S&T  program.  It  does  not  appear  that  mechanisms 
are  in  place  in  MDA  to  allow  MDA/AS  to  provide  this  support  to  the  program  Elements. 
Mechanisms  to  link  the  research  being  conducted  with  work  underway  in  the  Elements 
and  involving  them  in  the  long-term  S&T  program  will  have  to  be  established.  At  the 
same  time,  Element  involvement  must  not  divert  too  much  funding  away  from  the  long- 
tenn  S&T  objectives.  Continued  support  for  a  long-term  S&T  program  from  upper 
management  and  portfolio  management  that  commits  funding  to  long-term  research  will 
help  prevent  this. 

Seek  and  Obtain  Outside  Technical  Advice  and  Technologies — MDA  must  be 
able  to  take  advantage  of  the  advice  of  technical  experts,  as  well  as  technological 
advances  taking  place  throughout  the  DoD,  other  government  agencies,  domestic 
industry,  and  ultimately  the  world.  For  this  to  happen,  mechanisms  must  be  in  place  to 
keep  MDA  informed  of  technological  advances,  and  a  network  has  to  be  established  with 
other  technology  leaders  that  would  allow  MDA  to  take  advantage  of  technologies  that 
can  help  meet  its  mission.  In  addition,  mechanisms  to  incorporate  those  technologies  into 
future  MDA  systems  must  also  be  established. 

Establish  Partnerships  with  Technology  Leaders  Worldwide — Partnering  with 
other  technology  developers  is  a  method  of  networking  and  sharing  the  cost  of  research, 
accelerating  the  advancement  of  technologies,  and  steering  technology  developments  to 
meet  a  specific  MDA  application. 

In  addition,  internal  communications  is  critical  for  project  identification  and 
selection,  transition  of  technology,  and  maintaining  support  for  a  long-term  S&T  program 
from  both  MDA  leadership  and  the  Elements.  Fonnal  and  infonnal  mechanisms  to 
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increase  the  flow  of  information  to  the  Elements  and  MDA  Leadership  must  be 
established  if  a  healthy  long-term  S&T  program  is  going  to  be  maintained  in  MDA. 

Table  IV-2  and  Table  IV-3  list  specific  networking  and  outreach  mechanisms 
identified  through  the  case  studies.  Those  that  best  fit  MDA  and  how  they  can  be  applied 
are  discussed  in  the  next  section. 


Table  IV-2.  Networking  Mechanisms 


SEARCH 


External 


NETWORKING  MECHANISMS _ 

_ Internal 

COMMUNICATE 


•  Identify  and  use  of  other  organizations’  technology, 
including  government  labs. 

•  Hunters  and  gathers  seek  out  and  promote  the  use  of 
external  concepts  and  technologies. 

•  Locate  labs  near  important  intellectual  centers. 

•  Have  Chief  Scientist  maintain  liaison  with  scientific 
community. 

•  Conduct  “Idea  safaris"  to  elicit  ideas. 

•  Visit  laboratories  around  the  world. 

•  Conduct  brainstorming  sessions  with  top  scientists. 

•  Review  university  research,  scholarship,  or  intern 
programs. 

•  Prepare  focused  Broad  Agency  Announcements. 

•  Use  business  units  to  help  collect  technologies. 

EVALUATE 

•  Continually  assess  competitors’  technologies. 

•  Evaluate  intelligence  reports. 

•  Establish  technical/scientific  advisory  panels. 

•  Conduct  technology  gap  assessments. 

•  Hold  annual  S&T  conference — These  should  be 
technically  oriented. 

EXECUTE 

•  Partnering/Teaming. 

•  Share  cost  of  research  across  organizations. 

•  Establish  alliances  between  university  research  and 
commercial  firms. 

•  Have  joint  projects  with  customers. 

•  Spend  time  in  the  field  to  experience  problems  faced  by 
the  troops. 

•  Have  temporary  exchanges  of  technical  personnel 
among  DoD  and  industry  labs. 

•  Establish  Centers  of  Excellence  in  key  technologies. 


•  Facilitate  participation  of  business  unit  personnel  in 
central  research  programs. 

•  Have  Chief  Technology  Officer/Chief  S&T  officer  sit  on 
corporate  board. 

•  Corporate  Executive  Council  sets  S&T  strategy  through 
annual  process. 

•  Assign  personnel  to  select  customers  or  PM  offices. 

•  Reassign  individuals  between  R&D  to  assist  in 
transition. 

•  Hold  annual  internal  technical  conference(s). 

•  Hold  seminars  and  educational  programs. 

•  Create  an  internal  Web  site  for  research  to  collaborate 
on  new  ideas. 

•  Use  cross-disciplinary  teams,  Peer-to-peer  virtual 
networking,  and  regular  meetings  to  discuss  problems 
and  progress. 

•  Use  an  advocacy  program  at  small  business  units  to  be 
consultants  and  to  bring  ideas  back. 

EVALUATE 

•  Establish  strategic  technology  advisor  committee. 

•  Establish  technology  assessment/oversight/evaluation 
groups. 

•  Poll  engineers  annually  on  technology  needs  to  identify 
gaps. 

•  Establish  ad-hoc  technical  review  group. 

•  Co-locate  S&T  with  small  business  units  when  possible 
to  enhance  communications. 

GUIDE 

•  Involve  development  groups  in  S&T  management. 

•  Develop  road  maps  to  identify  gaps  and  overlaps  and  to 
determine  where  to  focus  S&T. 

•  Tie  S&T  to  corporate  strategy. 

•  Use  senior  S&T  managers/researchers  as  liaison  to 
business  units. 

•  Have  technology  steering  groups. 

•  Share  ideas  across  business  units. 

•  Identify  projects  with  development  groups  to  solve 
today’s  problems  and  meet  future  needs. 

•  Create  a  Component  Improvement  Program  that  is 
funded  to  correct  problems  identified  by  business  units. 

•  Build  close  relationships  with  PMs  to  determine  near- 
and  long-term  needs  and  to  help  it  shape  its  S&T 
program  to  meet  needs  of  customers. 

•  Small  business  units  provide  part  of  central  research 
funding  and  tell  them  what  they  need  done.  Details 
worked  through  negotiation. 
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_ NETWORKING  MECHANISMS _ 

_ External _ Internal _ 

•  Central  research  should  work  with  customers  to  identify 
1 00  plus  key  objectives  at  beginning  of  year  that  formed 
basis  for  funding. 

•  Do  cost  sharing/partnering  with  development  groups. 

•  Focus  on  transition  of  technology  to  development 
groups. 

•  Form  cross-functional  teams — Commodity  Management 
Approach. 

Table  IV-3.  Outreach  Mechanisms 
OUTREACH  MECHANISMS 

_ External _ Internal _ 

•  Flost  annual  S&T  conferences  that  are  technically  •  Maintain  top  level  support, 

oriented. 

•  Review  technical  journals.  •  Ensure  a  close,  continuing  relationship  between  the 

Chief  Executive  Officer  and  Chief  Technology  Officer. 

•  Obtain  support  of  leading  scientists  in  the  nation  and  •  Ensure  that  Technical  director  maintains  influence  over 

fund  their  ideas.  course  of  all  R&D. 

•  Provide  validation  of  scientific  reasonableness  to  public,  •  Science  Center  director  is  required  to  provide  regular 

administration,  and  congress.  “State  of  Science”  reports  to  the  Board. 

•  Attract  best  and  brightest  scientists  to  maintain  interest  •  Maintain  product  division  support, 
and  participation  from  scientific  community. 

•  Offer  continual  field  demonstrations  for  Congress,  •  Head  of  Science  Center  personally  represents  the 

headquarters,  and  users.  Center  to  the  operating  divisions  at  the  highest  possible 

level. 

•  Closely  cooperate  with  Army  staff  and  field  •  Solve  business  unit  problems  to  demonstrate 

commanders.  importance  of  central  R&D. 

•  Lead  and  actively  participate  in  meetings  and  •  Science  Center  must  be  viewed  as  an  important  asset, 

conferences. 

•  Have  temporary  exchanges  of  technical  personnel  •  Hold  corporate  technical  conferences, 

among  Army,  DoD,  and  industry  labs. 

•  Spend  time  in  the  field  to  experience  problems  faced  by  •  Use  metrics  to  measure  small  business  unit  and  central 

troops.  research  performance. 

•  Support  outside  publications.  •  Establish  metrics  for  projects  and  overall  program. 

•  Get  out  and  spend  time  with  customers,  users,  •  Review  technical  effectiveness  of  business  units  to 

scientists,  and  government.  determine  S&T  resource  allocation. 

•  Small  business  units  grade  corporate  research  and 
development  annually  on  whether  it  is  meeting  key 
objectives. 

•  Document  and  advertise  successes  in  organization. 

•  Create  internal  journals,  (e.g.,  Science  at  Rockwell). 

IMPLEMENTATION 

Implementing  the  changes  required  to  establish  a  centrally  controlled  long-term 
S&T  program  in  MDA  will  be  difficult  because  of  the  organization’s  current  focus  on 
fielding  an  initial  missile  defense  capability.  The  approach  being  proposed  is  to  lay  out 
the  long-tenn  goals  and  start  with  some  initial  steps  that  can  be  taken  to  initiate  the 
process.  This  will  require  the  strong  support  from  MDA  leadership  and  will  likely  be  met 
with  resistance  from  the  Elements.  However,  including  the  Elements  in  the  process  and 
minimizing  resource  requirements  in  the  early  years  may  reduce  this  resistance.  In 
addition,  showing  early  success  (without  a  sacrificing  long-tenn  focus)  will  demonstrate 
value  to  the  organization. 
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Outlined  below  are  recommended  steps  for  MDA/AS  to  establish  internal  and 
external  networking  and  outreach  processes  to  support  the  establishment  of  a  long-term 
S&T  program  in  MDA.  These  initial  steps  begin  to  put  into  place  many  of  the  key 
attributes  discussed  above.  In  combination  with  the  management  initiatives  previously 
discussed,  these  steps  will  help  establish  a  long-tenn  S&T  program  in  MDA/AS  that, 
with  some  initial  successes  and  the  continued  support  of  MDA’s  leadership,  can  grow. 
The  plan  outlined  below  is  intend  to  be  implemented  over  a  3 -year  period  and  includes 
the  initial  steps  that  could  be  taken  to: 

•  Sustain  support  for  a  long-tenn  S&T  program  from  MDA’s  Command  Group 
and  establish  mechanisms  to  link  and  involve  the  Elements  in  process. 

•  Seek  and  obtain  technical  advice  from  outside  sources. 

•  Keep  abreast  of  advances  in  strategic  technologies. 

•  Establish  partnerships  with  technology  leaders  outside  MDA. 

•  Support  innovative  promotion. 

Sustain  Support  for  a  Long-term  S&T  Program 

The  primary  initial  focus  should  be  on  development  and  communication  of  top 
management  support.  MDA/AS  should  brief  the  Director  and  senior  MDA  leadership  on 
proposed  plans  for  strengthening  the  long-term  research  program  in  MDA  and  obtain 
their  approval.  That  approval  should  come  in  the  fonn  of  a  memorandum  from  the 
Director  of  MDA  to  the  Elements  that  does  the  following: 

•  Provides  a  commitment  from  the  Director  to  strengthen  the  long-term  research 
program  in  MDA,  establish  the  initial  framework  for  the  program,  and  lay  out 
the  initial  steps  that  will  be  taken. 

•  Establishes  an  S&T  Management  Board  and  Corporate  Oversight  Board  to 
manage  the  program. 

•  Establishes  funding  goals  and  a  portfolio  approach  for  the  long-term  research 
program  to  allow  resources  to  be  committed  to  long-term  research  (10-plus 
years)  while  providing  support  for  the  organization’s  more  immediate  needs. 

•  Directs  Elements  to  identify  members  for  an  S&T  Management  Board  and 
work  with  MDA/AS  to  recommend  a  research  agenda  and  strategic 
technologies  that  should  be  supported  and  funded  by  a  central  research 
program. 

•  Directs  that  the  Corporate  Oversight  Board  be  briefed  within  4  months  on  the 
proposed  central  research  agenda  and  on  how  S&T  Management  Board  will 
continue  to  support  a  strong,  centrally  funded  research  program. 
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Once  this  management  structure  is  approved,  networking  and  outreach  activities 
should  be  initiated  to  facilitate  the  exchange  of  information  in  MDA  about  the  S&T 
program  and  to  maintain  continued  support  for  the  program.  We  suggest  two  initial 
activities: 

Annual  MDA  S&T  Conference — This  would  be  a  1-  or  2-day  conference  for 
MDA  and  its  contractors.  The  primary  purpose  of  the  conference  would  be  an  exchange 
of  information  about  the  various  S&T  activities  underway;  it  would  also  provide  an 
opportunity  to  identify  potential  areas  of  cooperation  and  minimize  duplication  of  effort. 
The  conference  would  facilitate  transition  by  increasing  (1)  the  awareness  of  research 
underway  in  MDA,  (2)  the  identification  of  new  applications,  and  (3)  the  potential 
redirection  of  research.  In  addition  to  presentations  from  MDA/AS  on  the  current  S&T 
program,  the  conference  would  include  presentations  from  the  Elements  on  their  current 
activities  as  well  as  their  technology  needs.  The  extent  of  industry  participation  is 
something  that  would  have  to  be  worked  out  with  MDA,  but  at  a  minimum  they  should 
be  allowed  to  have  exhibits. 

Quarterly  Technology  Seminars — These  would  be  half-day  seminars  with  the 
primary  purpose  being  an  exchange  of  information  on  current  research  in  a  specific 
technology  area — more  than  likely  one  of  the  strategic  technologies  being  pursued  by 
MDA/AS.  Although  the  focus  of  the  presentations  would  be  on  MDA/AS  activities  in  the 
technology  area,  presentations  could  also  be  made  by  other  DoD  or  government  agencies, 
as  well  as  the  Elements,  industry  members,  or  academics. 

Additional  activities  that  could  begin  sequentially  or  concurrently,  depending  on 
resources,  are  (1)  developing  an  intranet  for  MDA  to  capture  and  make  available  the 
principal  results  and  data  from  the  S&T  activities  underway  in  MDA  and  (2)  establishing 
periodic  meetings  between  the  director  of  MDA/AS  and  the  Element  heads  to  review 
current  S&T  activities  underway  supporting  the  Elements,  issues  or  problems,  and 
research  priorities.  The  importance  of  meeting  with  the  Elements  to  discuss  issues  and 
review  priorities  was  highlighted  in  several  of  the  Phase  I  case  studies.  We  recommend 
beginning  during  the  third  year  because  there  will  already  have  been  significant 
interchange  during  the  first  two  years  to  establish  the  program  and  the  initial  research 
agenda. 

Seek  and  Obtain  Technical  Advice  from  Outside  Sources 

MDA/AS  should  seek  the  advice  of  outside  experts  to  validate  its  research 
agenda,  explore  alternative  approaches,  and  gain  support  from  these  experts  (who  in 
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some  cases  will  be  MDA  stakeholders)  for  MDA’s  long-term  S&T  program.  Many  of  the 
activities  described  in  this  chapter  will  support  this  objective,  but  several  steps  can  be 
taken  with  this  objective  specifically  in  mind. 

An  initial  step  would  be  to  establish  an  S&T  Advisory  Board  of  outside  experts 
on  ballistic  missile  defense  acquisition,  development,  and  technology,  with 
representatives  from  government,  industry,  and  academia.  The  board  could  meet  twice  a 
year  to  review  MDA’s  current  technology  challenges  and  MDA/AS’s  research  agenda 
and  to  provide  technical  advice  to  the  Director  of  MDA/AS  and  the  Corporate  Oversight 
Board.  The  S&T  Advisory  Board  would  have  a  core  group  of  members  that  could  be 
supplemented  with  technical  experts  based  on  the  current  challenges  and  technology 
thrusts  being  faced  by  MDA/AS. 

Another  step  would  be  to  establish  a  Web  page  for  individual  strategic 
technologies.  An  example  is  the  Web  site  developed  and  maintained  by  the  Institute  for 
Defense  Analyses  to  provide  cognizance  of  MEMS  technology  for  DARPA.  When 
combined  with  other  networking  activities,  a  quality  Web  page  that  the  scientific 
community  can  depend  on  for  reliable  infonnation  on  a  specific  technology  can  be  a 
valuable  tool.  Being  the  host  of  such  a  site  provides  significant  advantages  and  is  a 
valuable  step  in  networking  with  the  scientific  community. 

Keep  Abreast  of  Advances  in  Strategic  Technologies 

A  primary  function  of  networking  is  to  remain  cognizant  of  technological 
advances  taking  place  outside  of  MDA.  It  is  not  practical  to  attempt  to  do  this  for  all 
technologies  or  even  all  those  that  may  have  an  impact  on  MDA.  Instead,  we  recommend 
that  mechanisms  be  established  to  remain  cognizant  of  only  a  select  set  of  strategic 
technologies. 

First,  to  provide  the  technical  competency  needed  to  maintain  cognizance  of  the 
strategic  technologies,  it  may  be  necessary  initially  to  outsource  this  activity  to  a  cadre  of 
technical  experts  from  an  external  S&T  organization  that  will  be  able  to  interact  with  the 
scientific  community  worldwide  and  keep  them  abreast  of  technical  advances.  Second, 
MDA  should  investigate  the  use  of  a  smart  search  engine  that  can  perform  focused 
searches  for  specific  technologies  across  government,  industry,  university,  and  scientific 
journal  Web  sites.  Many  products  on  the  market  can  be  used,  but  they  have  different 
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levels  of  sophistication.2  The  actual  search  function  could  be  performed  by  and  FFRDC 
(federally  funded  research  and  development  center)  or  contractor  and  the  results  provided 
to  the  MDA/AS  staff  and  any  organization  or  individual  providing  technical  expertise  to 
MDA/AS. 

MDA  could  also  hold  technology  workshops  on  specific  technologies.  Although 
held  primarily  to  identify  partnering  opportunities,  these  workshops  could  also  be  used  to 
remain  cognizant  of  a  technology  area.  The  basic  model  being  suggested  are  the 
workshops  sponsored  by  the  Air  Force’s  Dual  Use  S&T  Program.  These  workshops  bring 
together  people  working  in  specific  technology  areas  to  identify  areas  of  common  interest 
and  explore  opportunities  to  work  together.  Because  they  are  fairly  labor  intensive 
activities,  holding  no  more  than  two  in  a  given  year  is  suggested. 

Building  on  this,  MDA  could  hold  its  own  Missile  Defense  Strategic  Technology 
conference.  This  would  be  an  intensive  undertaking,  but  it  would  facilitate  networking 
with  the  S&T  community  and  allow  MDA  to  bring  together  those  researchers  working  on 
the  technologies  of  most  interest  to  it.  The  technology  workshops  discussed  above  could 
be  an  integral  part  of  the  conference. 

Establish  Partnerships  with  Technology  Leaders 

Establishing  partnerships — jointly  funding  research  projects — is  one  of  the  most 
significant  activities  that  MDA  can  take  to  leverage  research  underway  outside  of  MDA 
and  to  leverage  available  internal  S&T  funds.  Partnerships  can  be  established  with  other 
DoD  and  government  organizations  and  with  industry.  Before  partnerships  can  be 
formed,  however,  a  detailed  analysis  of  technology  needs  has  to  be  accomplished.  This 
analysis  has  to  include  the  identification  of  gaps  in  research,  areas  of  mutual  interest,  and 
partnering  opportunities. 

A  first  step  would  be  to  hold  meetings  with  S&T  counterparts  in  DoD  and  other 
government  agencies  to  identify  interest  and  explore  opportunities  for  partnering.  These 
should  be  held  after  the  strategic  technologies  have  been  identified.  The  purpose  of  these 


-  They  primarily  fall  into  two  categories.  Inexpensive  software  that  can  download  vast  amounts  of  data, 
requiring  extensive  manpower  to  sort  through  and  extract  those  pieces  of  data  needed.  These  can  range 
in  price  from  less  than  $100  to  several  hundred  dollars.  The  second  option  is  more  expensive  software 
($100,000  to  $200,000)  that  performs  a  more  intelligent  search,  catalogs  what  has  been  downloaded, 
and  requires  much  less  manpower  to  make  sense  of  what  has  been  found.  This  may  be  a  more  logical 
option  for  MDA/AS.  The  initial  costs  should  more  than  pay  for  themselves  in  reduced  manpower 
requirements.  In  either  case,  a  thorough  review  of  options  and  a  cost-benefit  analysis  is  needed  before 
expending  resources  on  any  software  to  search  the  Internet. 
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meetings  is  to  determine  interest  and  to  develop  an  initial  plan  on  how  to  proceed.  A 
series  of  meetings  may  be  required  to  develop  the  relationships  needed  to  form  research 
partnerships. 

Following  this,  a  detailed  analysis  of  each  strategic  technology  will  have  to  be 
accomplished  before  any  serious  partnering  discussions  can  be  held.  For  instance,  in  the 
course  of  the  study  underlying  this  report,  nanotechnology  was  selected  as  technology 
that  may  be  of  interest  to  MDA,  and  it  served  as  an  example  of  what  could  be  done  for 
each  of  MDA’s  strategic  technologies.  The  analysis  of  nanotechnology  is  in  Appendix  B. 
It  includes  the  following: 

•  Current  research  underway  and  a  list  of  government  organizations  performing 
research  in  nanotechnology. 

•  Identification  of  key  research  areas  of  interest  for  MDA. 

•  Opportunities  for  MDA  to  network  with  organizations  performing  research  in 
nanotechnology. 

•  Specific  steps  MDA  can  take  to  become  engaged  in  nanotechnology  research. 

The  actual  analysis  of  the  strategic  technologies  would  have  more  detail  than 
what  was  done  for  nanotechnology,  and  it  should  include  gap  analysis  of  the  research 
underway  at  MDA  and  within  the  government  to  determine  those  areas  where  MDA  may 
want  to  fund  or  form  partnerships  with.  MDA  should  focus  its  funding  in  (1)  areas  that 
have  the  potential  of  providing  significant  benefit  to  MDA  that  are  currently  underfunded 
and  (2)  those  areas  being  adequately  funded  but  heading  in  a  direction  that  will  not 
support  future  MDA  needs.  Once  this  analysis  is  completed  for  a  strategic  technology, 
detailed  discussions  and  technology  workshops  (discussed  in  the  previous  section)  can  be 
held  with  potential  partners.  The  objective  of  these  discussions  is  to  reach  a  joint  funding 
agreement  that  includes  the  level  of  funding  for  each  partner  and  specifies  who  will  lead 
the  research  effort. 

If  MDA/AS  is  going  to  actively  support  the  Elements,  partnerships  can  also  be 
initiated  with  the  Elements.  Activities  described  above  under  “Sustain  Support  for  a 
Long-term  S&T  Program”  would  be  the  initial  steps  to  providing  support  to  the  Elements 
and,  where  applicable,  to  fonning  partnerships  with  them  to  fund  research  projects. 

Industry  partnerships  can  also  be  investigated  at  the  same  time.  A  detailed 
analysis  of  needs  is  still  required,  but  to  form  industry  partnerships,  it  would  also  have  to 
be  determined  whether  the  technology  has  commercial  potential  and,  if  so,  whether  MDA 
would  be  willing  to  lose  at  least  partial  control  of  the  technology  developed.  If  all 
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conditions  are  met,  a  Broad  Agency  Announcement  could  be  released  for  select  research 
efforts  in  support  of  the  strategic  technologies,  which  possibly  would  meet  the  pressing 
needs  of  the  Elements. 

Support  Innovation  Promotion 

The  activities  described  in  the  preceeding  sections  will  establish  the  type  of 
organization  needed  to  support  a  viable  long-term  research  program  and  foster  innovation 
promotion.  In  addition,  the  networking  and  outreach  activities  described  above  will  result 
in  a  substantial  flow  of  information  that  will  help  identify  and  analyze  innovative 
concepts.  Chapter  II  includes  a  detailed  discussion  of  networking  and  outreach  in  the 
service  of  innovation  promotion. 

CONCLUSION 

Active  networking  with  the  scientific  community  inside  and  outside  MDA  is 
crucial  to  identifying  promising  technologies,  potential  emerging  threats,  gaps  in  current 
research,  and  partnering  opportunities.  Outreach  to  these  communities  and  to  MDA 
executive  and  program  leaders  will  be  necessary  to  gaining  support  for  a  healthy  long¬ 
term  research  program.  This  chapter  has  described  numerous  steps  that  can  be  taken  by 
MDA/AS  over  the  next  3  years  to  establish  internal  and  external  networking  and  outreach 
processes  to  sustain  support  for  a  long-term  S&T  program,  seek  and  obtain  technical 
advice  from  outside  sources,  keep  abreast  of  advances  in  strategic  technologies,  establish 
partnerships  with  technology  leaders,  and  support  innovation. 
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V.  ANALYTICAL  TECHNIQUES  FOR  S&T  PLANNING  AND 

MANAGEMENT 


John  Meyer 


INTRODUCTION 

This  chapter  reviews  analytical  techniques  and  methods  that  can  be  used  to 
support  S&T  planning  and  management  within  MDA/AS  and  identifies  those  techniques 
that  should  be  given  the  highest  priority  for  application.  This  is  a  broad  survey  of 
techniques  used  within  government  and  industry  to  support  activities  such  as  research 
strategy  development,  generation  of  research  concepts,  project  definition  and  assessment, 
and  similar  research  planning  and  management  activities.  What  follows  is  summary  of 
the  tools  available,  a  discussion  of  the  most  suitable  tools  for  each  area  of  application, 
and  a  recommended  short  list  of  tools  and  techniques  MDA/AS  should  consider  for  near- 
term  implementation.  The  treatment  is  broad  but  relatively  shallow.  Thus,  the  results  can 
be  considered  an  initial  “directory”  of  tools  and  techniques  that  can  serve  as  a  source  for 
selecting  specific  methods  that  can  be  embedded  in  MDA/AS ’s  overarching  approach  to 
S&T  planning  and  management,  with  particular  emphasis  on  those  tools  that  can  add  to 
the  longer  term  aspects  of  the  innovation  process. 

The  planning  and  management  of  MDA’s  S&T  program  involves  a  wide  range  of 
functions,  including  activities  such  as: 

•  Environmental  scanning  to  maintain  an  awareness  of  changing  needs  and 
emerging  technologies. 

•  Development  of  research  strategies  to  guide  program  planners  and  managers. 

•  Determination  of  focus  areas,  be  they  key  technical  challenges,  specific 
promising  technologies,  or  important  solution  approaches. 

•  Identification  and  development  of  project  concepts  that  may  be  considered  for 
funding. 

•  Selection  of  specific  research  projects  to  be  pursued. 

•  Management  of  on-going  projects  to  bring  them  to  successful  completion  or 
tennination. 
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•  Transition  of  research  results  to  other  MDA  elements  or  other  users. 

•  Provision  of  infrastructure  capabilities  needed  to  support  the  research  program. 

Several  examples  of  these  types  of  functions  can  be  found  in  the  MDA  Phase  II 
pilot  study.1  As  part  of  this  pilot  effort,  the  project  team  defined  processes  for  identifying 
alternative  missile  defense  concepts,  developed  scoring  models  for  ranking  the  concepts, 
and  established  methods  for  mapping  systems  concept  technology  needs  to  core 
technologies.  Similarly,  a  method  of  linking  MDA  to  external  technology  sources  has 
been  identified  as  a  needed  capability  during  IDA’s  interaction  with  the  sponsor. 

A  large  and  rapidly  growing  group  of  tools  and  techniques  has  emerged  during 
the  past  few  decades  to  support  the  above-mentioned  S&T  planning  and  management 
functions  within  both  industry  and  government.  These  methods  range  from  high-level 
management  concepts,  like  the  use  of  stage-gate  project-management  frameworks  to 
oversee  a  collection  of  research  projects,  to  relatively  narrow  analytical  tools  for 
performing  specific  functions,  such  as  applying  qualitative  or  quantitative  scoring  models 
to  evaluate  and  select  individual  projects  for  funding.  Particularly  important  are 
advancements  in  tools  such  as  modeling  and  simulation  that  have  been  made  in  recent 
years  to  support  S&T  planning  and  management. 

IMPLEMENTATION  ENVIRONMENT  AND  OPTIONS 

Although  many  tools  and  techniques  exist  to  support  S&T  management  functions, 
a  significant  number  are  not  applicable  to  MDA’s  situation  or  would  require  considerable 
modification  from  their  original  form  to  satisfy  MDA’s  needs.  For  example,  many 
commercial  tools  and  techniques  focus  heavily  on  financial  parameters,  such  as  return  on 
investment,  or  on  market  share  data  as  a  basis  for  analysis  and  decision-making.  Such 
techniques  would  have  limited  utility  to  MDA  or  would  need  to  be  reworked  to 
incorporate  measures  of  effectiveness  relevant  to  MDA. 

Beyond  this  issue  of  user  focus  (i.e.,  defense  research  applications  versus 
commercial  product  development),  selection  of  MDA  research  planning  and  management 
tools  and  techniques  also  needs  to  take  into  consideration  a  number  of  other 
implementation  issues.  The  tools  and  techniques  should: 


1 


Richard  Van  Atta,  et  al.,  “Results  of  a  Technology  Assessment  Pilot  Project  for  the  Missile  Defense 
Agency,”  forthcoming. 
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•  Be  consistent  with  DoD  S&T  policies,  planning  procedures,  strategies,  and 
recent  changes  in  this  area,  such  as  the  newly  revised  DoD  5000-series 
regulations  and  instructions. 

•  Be  compatible  with  current  and  anticipated  MDA  research  planning  processes, 
and  appropriately  interface  with  budgetary  processes  and  the  generation  of 
such  documents  as  the  TOG  (Technical  Objectives  and  Goals)  document,  the 
Enhancement  Plan,  and  the  BMDS  Integrated  Program  Plan. 

•  Comply  with  established  DoD  norms  regarding  interfacing  with  the  private 
sector  from  an  information  sharing,  planning,  and  contracting  perspective. 

•  Take  into  consideration  staffing  procedures  and  possible  constraints  in  this 
area  or  provide  suitable  alternative  solutions  to  bring  the  appropriate  human 
resources  to  bear  on  the  problem. 

•  Recognize  the  different  stakeholders  that  will  participate  in  the  research 
planning  and  management  process  within  MDA,  DoD,  other  government 
agencies,  industry,  and  academia,  as  well  as  possibly  foreign  collaborators. 

A  number  of  the  tools  described  in  this  report  have  already  been  or  are  being  used 
to  some  extent  by  MDA/AS.  No  attempt  was  made  to  determine  how  the  techniques 
presented  herein  would  be  integrated  with  MDA/AS’ s  existing  methods.  It  is  assumed 
that  implementation  details  (e.g.,  priorities,  timing,  and  resources)  will  be  determined  in 
the  future.  Any  implementations  of  the  techniques  contained  in  this  report  would  likely  to 
be  incorporated  in  and  consistent  with  existing  MDA/AS  planning  and  management 
frameworks.2  Implementation  of  new  planning  and  management  tools  must  also  be 
pursued  in  a  manner  consistent  with  other  priorities  and  resource  limitations  within 
MDA.  Furthermore,  it  is  logical  to  assume  that  some  of  these  tools  and  techniques  should 
be  implemented  in  a  sequential  rather  than  a  parallel  manner  and  that  it  may  take  several 
years  to  achieve  implementation  of  the  entire  set  of  capabilities  envisioned. 
Implementing  the  new  S&T  planning  and  management  tools  could  be  accomplished 
through  a  combination  of  hiring  additional  MDA  staff  or  by  contracting  to  outside 
sources,  depending  upon  such  factors  as  timing,  resource  levels,  staffing  ceilings, 
suitability  for  outsourcing,  and  similar  factors.  It  may  also  be  desirable  to  contract  for 
some  functions  in  the  early  stages  of  implementation  and  then  migrate  to  in-house 
operations  at  a  later  date.  Thus,  an  overall  implementation  strategy  and  multiyear 
implementation  plan  will  most  likely  be  required. 


2  Existing  MDA/AS  planning  and  management  frameworks  are  described  in  several  documents, 
including  (1)  “The  Way  We  Do  Business  -  The  BMDA  Integrated  Program  Plan,”  MDA,  11  June 
2003  (Draft);  (2)  Block  Enhancement  Plan;  (3)  Technology  Enhancement  Plan;  (4)  System  Evolution 
Plan;  and  similar  publications. 
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DESCRIPTION  OF  SOLUTION  ELEMENTS 


A  variety  of  tools  and  techniques  have  been  identified  to  support  S&T  planning 
and  management  functions,  and  many  of  these  tools  will  support  more  than  one  function. 
More  than  40  different  analytical  tools  and  management  techniques  have  been 
identified.3  Many  of  the  most  relevant  tools  are  briefly  discussed  in  this  section;  all  of  the 
identified  tools  are  summarized  in  the  appendixes. 

For  discussion  purposes,  the  tools  have  been  organized  into  five  categories: 

•  Management  tools. 

•  Planning  tools. 

•  Assessment  tools. 

•  Survey  tools. 

•  Forecasting  tools. 

•  Other  tools  and  techniques. 

In  addition  to  identifying  individual  tools  and  techniques  during  the  task,  each 
was  characterized  in  terms  of  its  purpose  and  benefits  and  overall  approach.  These 
methods  are  summarized  by  category  in  the  appendixes. 

The  reader  should  note,  however,  that  the  tools  may  not  fall  cleanly  in  one 
category.  For  example,  the  technique  of  Delphi  forecasting,  in  which  experts  are  polled 
concerning  the  likelihood  and  timing  of  future  events,  can  be  viewed  as  a  survey  method, 
an  assessment  tool,  and  a  forecasting  process.  Thus,  the  category  a  specific  tool  or 
technique  has  been  placed  in  is  somewhat  arbitrary.  More  in-depth  discussions  of  how 
three  of  these  techniques  (stage-gate  processes,  scoring  models,  and  modeling  and 
simulation)  can  be  tailored  to  MDA/AS’s  needs  are  presented  in  later  sections  of  the 
report. 

Management  Tools 

A  number  of  management  concepts  have  been  developed  to  support  the  planning 
and  execution  of  research  programs.  A  number  of  these  methods  were  uncovered  during 
the  Phase  I  study,  and  several  of  these  (i.e.,  the  use  of  key  technology  steering  groups, 


3  The  definition  of  an  “analytical”  tool  or  technique  is  somewhat  arbitrary  and  subjective.  An  attempt 
was  made  to  include  those  methods  that  are  recognized  as  supporting  S&T  planning  and  management 
functions.  If  there  was  some  question  about  whether  a  method  is  “analytical,”  we  erred  on  the  side  of 
inclusion.  Thus,  all  techniques  presented  in  this  report  may  not  be  viewed  as  being  analytical  by  all 
readers. 
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innovation  hubs,  and  networking  and  outreach  techniques)  have  been  explored  in  more 
detail  in  chapters  of  this  report. 

Management  concepts  to  support  research  planning  and  execution  cover  a  wide 
range  of  techniques,  from  broad  frameworks  to  narrowly  defined  approaches  focusing  on 
specific  aspects  of  research  management.  Some  of  the  more  relevant  concepts  for  MDA’s 
needs  include  the  following: 

Early  stage-gate  processes — A  stage-gate  process  is  a  structured  framework  for 
managing  R&D  projects,  where  the  stages  represent  the  phases  or  steps  the 
project  must  progress  through,  and  the  gates  refer  to  review  points  or  intermediate 
milestones  where  the  progress  and  future  direction  of  the  project  are  reviewed 
against  a  set  of  previously  defined  criteria.  Stage-gate  processes  are  widely  used 
in  industry  to  support  commercial  product  development,  and  several  proprietary 
versions  of  the  technique  are  available  to  speed  implementation.  Stage-gate 
systems  are  typically  used  to  (1)  guide  decisions  on  which  project  to  fund;  (2) 
align  projects  with  R&D  and  organizational  objectives;  (3)  provide  guidance  on 
project  definition,  including  scope,  desired  outputs,  integration,  and  transition  of 
results;  and  (4)  review  projects  to  ensure  progress,  programmatic  fit,  and  priority. 
Perhaps  one  of  the  best  examples  of  a  stage-gate  technique  is  the  DoD  weapon 
system  development  process,  which  has  highly  structured  milestones  and 
intennediate  RTD&E  activities. 

Some  users  have  observed  that  a  stage-gate  process  is  best  suited  for  managing 
well-defined  product  development  efforts  and  that  it  does  not  handle  the  early 
“fuzzy  front  end”  of  R&D  very  well.  However,  a  number  of  organizations  have 
successfully  addressed  this  criticism  by  modifying  the  technique  to  encourage 
innovation  in  the  early  stage  of  the  process  while  still  maintaining  a  structured 
management  approach.  One  variation  of  this  approach  is  referred  to  as  the 
technology  stage-gate  method,  which,  for  example,  has  been  implemented  by 
DuPont  as  its  APEX  process. 

A  similar  but  customized  process  to  define  and  manage  early  research  and 
technology  projects  would  appear  to  be  an  essential  need  for  MDA.  Such  a 
process  would  be  designed  to  feed  new  component  technologies  and  systems 
concepts  into  MDA’s  overall  research,  development,  test,  and  evaluation 
(RDT&E)  pipeline  and  provide  the  necessary  internal  and  external  linkages  and 
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integration  to  ensure  success.  (A  stage-gate  process  for  MDA/AS  is  discussed  in 
more  detail  later  in  this  chapter.) 

Benchmarking  and  performance  metrics — The  use  of  benchmarks  and 
performance  metrics  to  evaluate  research  management  is  becoming  increasingly 
common,  particularly  in  industry.  A  number  of  leading  research  organizations 
have  made  the  use  of  benchmarking  and  perfonnance  metrics  an  integral  part  of 
their  management  approach  to  ensure  they  are  doing  everything  possible  to 
increase  performance  and  satisfy  the  needs  of  their  parent  organization. 
Sometimes  benchmarking  is  also  used  to  compare  the  performance  or  capabilities 
of  different  technologies,  products,  or  research  facilities  to  determine  which 
represent  state-of-the-art,  or  world  class,  levels  of  accomplishment. 

Portfolio  management — This  is  a  relatively  straightforward  but  important 
management  tool  that  helps  ensure  balance  among  various  types  of  projects 
within  a  research  program.  Such  an  approach  would  typically  establish  minimum 
funding  levels  for  categories  of  projects.  These  categories  might  represent 
technical  risk  (high  versus  low  risk),  application  time  frame  (long-range  versus 
near-tenn),  type  of  innovation  (radical  versus  conventional),  targeted  end-use 
applications  (boost-phase  intercept  versus  tenninal-phase  intercept),  technologies 
(sensors  versus  integrated  battle  management),  or  type  of  project  (applied  versus 
fundamental  research).  Using  this  approach,  a  minimum  level  of  aggregate 
funding,  for  example,  would  be  specified  for  key  technologies,  radical  innovation 
projects,  or  projects  supporting  existing  program  elements,  thereby  ensuring  that 
advances  are  pursued  in  each  of  these  areas. 

Portfolio  management  is  also  used  for  periodically  evaluating  individual  research 
projects  within  a  category  to  ensure  that  each  project  is  handled  appropriately, 
whether  it  be  accelerated,  scaled  back,  or  terminated  based  on  the  latest 
information,  and  is  strategically  aligned  with  current  organizational  priorities. 

Strategic  technology  alliances — Formal  collaboration  mechanisms  are  being 
increasingly  used  by  research  organizations  to  extend  their  technology  coverage 
and  improve  their  resource  effectiveness.  In  the  case  of  MDA,  this  could  prove  to 
be  an  effective  management  strategy  by  leveraging  ongoing  research  being 
pursued  by  other  DoD,  Federal,  and  nongovernment  programs. 

Cross-functional  teams — Cross-functional  teaming  arrangements  are  routinely 
used  today  to  define  and  manage  research  projects,  to  bring  broad  expertise  and 
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perspectives  to  bear  during  early  definitional  stages,  and  to  speed  and  smooth  the 
transition  of  results.  How  cross-functional  teams  are  planned,  formed,  and 
managed  can  have  a  major  affect  on  the  success  of  the  research  project.  Cross¬ 
functional  teams  are  particularly  relevant  for  research  in  technologies  that  will  be 
used  in  complex  systems,  such  as  those  developed  by  MDA. 

The  above  represents  a  few  of  the  management  tools  and  techniques  that  can  be 
used  by  MDA/AS  to  plan  and  manage  its  S&T  program.  Additional  details  on  these  tools 
are  presented  in  Appendix  C,  Table  C-l,  along  with  summaries  of  several  other  methods. 

Planning  Tools 

Research  planning  methods  are  generally  used  to  produce  fundamental  decisions 
and  actions  that  shape  and  guide  what  a  research  organization  is  and  does.  Such 
techniques  are  typically  used  to  (1)  turn  mission  and  vision  statements  into  actions, 
(2)  bring  structure  and  measurement  to  research  planning,  (3)  develop  strategic  priorities, 
(4)  communicate  research  priorities  and  guide  decision-making,  and  (5)  assess  future 
scenarios  and  select  strategies  to  deal  with  those  outcomes. 

Although  strategic  planning  is  practiced  in  most  large  research  organizations,  it  is 
not  universally  loved  and  respected.  Based  on  at  least  several  decades  of  practical 
experience,  a  number  of  observers  question  whether  strategic  planning  is  worth  the 
effort: 


A  good  deal  of  corporate  planning  I  have  observed  is  like  a  ritual  rain 
dance;  it  has  no  effect  on  the  weather  that  follows,  but  those  who  engage 
in  it  think  it  does.  Moreover,  much  of  the  advice  and  instruction  related  to 
corporate  planning  is  directed  at  improving  the  dancing,  not  the  weather.4 

However,  others  feel  equally  as  strongly  that  strategic  planning  is  a  desirable  and 
essential  tool  that  needs  to  be  used  by  all  organizations,  including  research  groups.  This 
is  based  on  the  belief  that  the  insight  and  consensus  gained  through  proper  strategic 
planning  far  outweigh  the  adverse  effects  of  inaccurate  plans.  If  nothing  else,  strategic 
planning  forces  an  organization  to  address  risks  and  internal  and  external  trends,  thus 
enabling  a  faster,  more  appropriate  response  to  changes. 

Regardless  of  where  one  stands  on  this  philosophical  debate,  the  following  points 
are  clear  with  respect  to  strategic  planning  for  research  programs: 


4  Russell  Ackoff,  Creating  the  Corporate  Future,  John  Wiley  &  Sons,  1981. 
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•  Strategic  planning  is  not  the  same  as  forecasting  or  predicting  the  future,  and 
should  not  be  viewed  as  guaranteeing  success,  only  increasing  its  likelihood. 

•  Strategic  planning,  particularly  in  support  of  research  programs,  needs  to 
carefully  balance  a  top-down  analysis  of  trends  and  issues  with  the  bottom-up 
generation  of  new  ideas  in  a  way  that  does  not  stifle  major  innovations. 

•  Research-related  strategic  planning  should  explicitly  take  into  consideration 
the  possibility  of  radical  changes  in  technologies,  threats,  product  features,  and 
other  underlying  factors — be  they  internally  or  externally  generated — that 
could  dramatically  alter  demand  for  the  parent  organization’s  products.5 

Since  MDA  already  has  a  strategic  planning  system  (as  outlined  in  the  BMDS 
Integrated  Program  Plan),  the  issue  is  not  whether  such  planning  should  be  undertaken. 
Rather,  it  is  a  question  of  how  planning  for  research  programs  should  be  performed  and 
interfaced  with  MDA’s  overall  strategic  plan.  A  few  of  the  planning  techniques  that 
could  prove  useful  in  this  regard  are  briefly  mentioned  below. 

SWOT  (strengths,  weaknesses,  opportunities,  threats)  analysis — This  is  a 
widelyused  technique  for  examining  how  an  organization  can  perform  relative  to 
its  mission,  capabilities,  and  environment.  The  SWOT  technique  provides  an 
overall  view  of  the  organization  and  the  factors  that  affect  its  performance, 
establishes  a  baseline  for  creating  strategies  that  address  key  issues,  and  points  to 
critical  issues  that  must  be  addressed  if  the  organization  is  to  succeed.  This  tool  is 
often  used  as  part  of  a  traditional  strategic-planning  system. 

Balanced  scorecard — This  is  a  planning  and  performance  measuring  technique 
that  is  often  used  by  commercial  organizations.  It  traditionally  focuses  on 
balancing  four  performance  dimensions:  (1)  customer  perceptions  of  how  the 
organization  is  perfonning;  (2)  internal  perceptions  of  how  the  organization  is 
doing  and  what  it  must  excel  at;  (3)  innovation  and  learning  performance;  and  (4) 
financial  performance.  In  each  dimension,  appropriate  goals  and  perfonnance 
metrics  are  established  and  tracked.  What  makes  this  technique  attractive  to  users 
is  the  overall  balance  it  brings  to  planning  and  execution,  which  if  properly 
structured  keeps  the  organization  focused  on  those  activities  that  truly  affect  its 
success.  Obviously,  to  be  of  use  to  MDA/AS,  the  performance  dimensions  and 
resulting  goals  and  metrics  would  need  to  be  customized  to  MDA/AS ’s  specific 


5  Motorola  refers  to  this  as  being  sensitive  to  the  “gotchas”  that  represent  research  leading  to  innovative 
products  or  new  product  features  that  can  quickly  change  market  dynamics.  Some  of  these 
technologies  can  be  developed  by  competitors,  while  others  may  be  produced  internally  by  Motorola. 
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research  circumstances,  strategies,  and  priorities,  rather  than  addressing  overall 
MDA  issues. 

Technology  road  mapping — Road-mapping  techniques  cover  a  wide  range  of 
planning  methods  that  have  been  used  with  varying  degrees  of  success  to  define 
long-term  research  programs.  What  most  of  these  road-mapping  techniques  have 
in  common  is  the  use  of  some  type  of  graphical  representation  of  technology 
evolution  or  technology  plans  mapped  against  time.  Such  a  map  is  used  to  guide 
research  and  new  technology  development  or  for  selecting  technologies  to  be 
used  in  long-term  new  product  development. 

However,  most  road-mapping  exercises  involve  much  more  than  just  the 
generation  of  a  graphical  map.  They  often  start  with  a  long-term  vision  for  a 
technology  or  product.  This  vision  is  compared  with  the  current  state  of  the  art  to 
identify  technology  gaps  or  voids  that  need  to  be  addressed  and  technical  barriers 
that  need  to  be  overcome  to  achieve  the  vision.  The  research  needed  to  fill  these 
gaps  and  overcome  the  barriers  is  then  identified  and  laid  out  in  a  time-based 
program  to  produce  the  road  map. 

Although  road  mapping  can  be  a  useful  planning  tool,  it  must  be  used  cautiously 
to  avoid  several  common  pitfalls.  Unfortunately,  road  mapping  has  gained  a  poor 
reputation  in  some  instances  because  it  became  an  end  rather  than  a  means  for 
S&T  planning  and,  in  some  cases,  stifled  innovation  due  to  political  factors  or  too 
restrictive  a  view  of  what  was  technically  possible.  Thus,  road-mapping  efforts 
must  be  carefully  planned  and  managed  to  ensure  they  do  not  distort  the  S&T 
program  in  undesirable  ways. 

One  of  the  best  examples  of  an  effective  road  mapping  technique  is  the  GOTChA 
(Goals,  Objectives,  Technical  Challenges,  Approaches)  methodology  that  has 
been  used  successfully  for  the  IHPTET  (Integrated  High  Performance  Turbine 
Engine  Technology)  program  in  the  1990s  and  that  is  now  being  applied  to  the 
Future  Combat  Systems  and  VAATE  (Versatile  Affordable  Advanced  Turbine 
Engine)  programs.  GOTChA  is  a  planning  technique  that  maps  systems-level 
performance  and  technology  goals  to  R&D  activities  needed  to  achieve  the  goals. 
The  technique  provides  a  framework  for  determining  goal-directed  S&T  program 
requirements.  GOTChA  helps  ensure  that  S&T  funds  are  spent  in  a  focused  and 
productive  manner  and  provides  a  tool  for  monitoring  progress  of  individual 
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technology  efforts.  In  many  respects,  GOTChA  is  a  quantitative  technology  road¬ 
mapping  technique. 

An  example  of  a  high-level  GOTChA  chart  for  the  Anny’s  Future  Combat 
System  is  shown  in  Figure  V-l.  This  example  illustrates  how  high-level  systems 
goals  are  broken  down  into  more  focused  objectives,  technical  challenges,  and 
solution  approaches. 

These  and  other  planning  tools  and  techniques  are  summarized  in  Appendix  C, 
Table  C-l. 


IDA 

GOALS 


SOA:  State  of  Art 


IDA 


Figure  V-1.  Example  GOTChA  Chart  for  the  Army’s  Future  Combat  System 
(Source:  Donald  Dix,  “The  GOTChA  Methodology,”  IDA,  15  May  2002) 


Assessment  Tools 

Assessment  tools  are  used  for  a  variety  of  reasons  in  planning  and  managing 
research  programs.  For  example,  assessment  models  can  be  used  to  evaluate  individual 


V-10 


projects  so  that  the  most  suitable  or  highest  priority  can  be  selected  for  funding.  Other 
types  of  assessment  models  may  be  used  to  evaluate  the  impact  of  new  technologies  on 
product  performance  or  to  identify  areas  (e.g.,  subsystems  or  components)  having  the 
greatest  potential  for  technical  improvement.  Still  other  types  of  assessment  techniques 
can  be  used  to  decompose  problems  or  challenges  in  a  way  that  facilitates  the 
identification  of  appropriate  solutions.  And  still  other  types  of  assessment  models,  such 
as  the  DoD  Technology  Readiness  Levels,  are  used  to  determine  the  maturity  of 
technologies  and  their  readiness  to  move  down  the  product  development  pipeline. 

A  large  number  of  assessment  tools  have  been  developed  to  support  research 
activities.  A  few  of  the  more  relevant  to  MDA/AS’s  needs  include  the  following: 

Simulation  models — Simulation  models  are  widely  used  to  assess  the  impact  of 
new  technologies  on  the  performance  of  products  without  the  need  to  construct 
and  test  physical  prototypes,  which  are  particularly  costly  in  MDA’s  case.  Such 
models  have  been  particularly  important  in  support  of  defense  and  aerospace 
product  developments.  One  example  highlighted  during  the  Phase  I  project  was 
the  role  the  Visionics  quantitative  performance  models  played  in  the  development 
of  night  vision  technology.  In  this  case,  the  Visionics  models  provided  objective 
criteria  for  assessing  system  and  component  performance.  The  use  of  these 
models  for  R&D  planning  and  assessing  options  before  investment  proved  to  be 
an  effective  means  of  allocating  resources. 

MDA  is  already  using  a  variety  of  simulation  models  to  assess  the  overall 
performance  of  missile  defense  systems  and  subsystems.  However,  additional 
models  to  support  analysis  of  key  technologies,  important  technical  challenges, 
and  emerging  solutions  will  undoubtedly  be  needed  in  the  future. 

Simulation  and  modeling  as  a  tool  for  use  by  MDA/AS  is  discussed  in  Chapter  VI 
of  this  report. 

Project  and  technology  assessment  tools — These  tools  are  needed  to  enable 
research  managers  to  objectively  evaluate  alternative  projects  and  technologies  in 
tenns  of  their  potential  contribution  to  future  MDA  systems.  These  assessment 
models  can  range  from  relatively  straightforward  checklists  to  ensure  that  major 
issues  have  been  addressed,  to  elaborate  multivariable  scoring  models  to  rank 
alternatives  in  a  more  quantitative  manner. 

Problem  assessment  techniques — A  large  number  of  assessment  techniques  exist 
for  analyzing  various  problems  or  technical  challenges.  These  vary  widely  in 
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complexity  and  include  techniques  such  as  affinity  diagrams,  cause-and-effect 
(fishbone)  diagrams,  conjoint  (trade-off)  analysis,  decision  trees,  gap  analysis, 
influence  diagrams,  morphological  matrices,  prioritization  matrices,  radar/spider 
charts,  scenario  analysis,  sensitivity  analysis,  and  strategy  space  models. 

Each  of  the  above  techniques  and  others  are  summarized  in  Appendix  C,  Table 
C-3.  Collectively,  these  represent  a  broad  spectrum  of  tools  that  MDA  research  managers 
can  draw  upon  to  assess  research  needs  and  options. 

Survey  Tools 

Few  research  organizations  could  operate  without  inputs  from  outside  the 
organization.  Information  from  customers  and  end  users,  other  elements  within  the 
organization,  and  outside  experts  are  used  to  identify  research  needs,  develop  new 
solution  ideas,  and  evaluate  alternative  project  concepts.  Some  of  the  survey  tools 
available  to  support  the  collection  and  analysis  of  this  type  of  information  include  the 
following: 

Structured  brainstorming — Brainstorming  is  a  method  of  creative  problem¬ 
solving  frequently  used  in  research  and  product  concept  generation.  There  are 
many  variations  of  the  brainstorming  format.  The  basis  of  all  these  methods  is  to 
use  a  group  of  appropriately  selected  people  to  creatively  generate  a  list  of  ideas 
related  to  a  particular  challenge  or  objective.  Some  brainstonning  processes  are 
highly  structured  to  increase  the  chances  of  yielding  radical  innovative  solutions. 
Keys  to  brainstorming  success  include  (1)  selecting  the  right  participants,  (2) 
keeping  the  process  focused  on  actionable  objectives,  and  (3)  facilitating  the 
interactions  in  a  manner  that  minimizes  negative  dynamics  among  the  participants 
that  would  limit  creativity.  In  addition  to  the  generation  of  innovative  new 
solutions,  other  brainstorming  benefits  include  (1)  improved  teamwork  and 
commitment  to  action,  (2)  reduced  fear  of  competition  and  overcoming  project 
obstacles,  (3)  creating  a  high  volume  and  wide  variety  of  actionable  ideas,  and 
(4)  increased  appreciation  for  nontraditional  thinking. 

Brainstorming  was  one  of  the  techniques  used  by  the  IDA  Phase  II  pilot  task  team 
for  initial  identification  of  innovative  missile  defense  concepts,  which  were  later 
detailed,  refined,  and  analyzed  to  identify  important  S&T  challenges. 

Brainstorming  techniques  are  often  employed  in  conjunction  with  the  types  of 
assessment  tools  discussed  in  the  previous  section;  here,  the  brainstorming 
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sessions  are  used  to  generate  many  ideas  and  the  assessment  tools  are  used  to 
evaluate,  rank,  and  refine  them. 

Nominal  group  techniques — These  represent  a  variety  of  methods  for  generating 
ideas  about  a  particular  subject  and  building  team  consensus  about  priorities. 
Although  similar  to  brainstonning,  the  emphasis  on  nominal  group  techniques  is 
to  ensure  equal  team  member  participation  in  identifying  and  ranking  of  issues 
and  solutions  and  build  team  commitment  to  the  outcome  of  the  process.  Whereas 
brainstorming  concentrates  on  generating  solutions,  nominal  group  techniques 
focus  on  achieving  consensus  results. 

Internet-based  surveys — The  Internet  is  increasingly  being  used  as  a  tool  for 
collecting  opinions.  These  collections  range  from  wide  surveys  of  inputs  from  any 
source  to  more  structured  collections  of  expert  opinion.  For  example,  some 
government  agencies  now  use  the  Internet  for  collecting  inputs  from  evaluation 
panels  who  are  reviewing  research  proposals.  Such  Internet-based  tools  will 
become  increasingly  important  in  the  future,  particularly  for  real-time, 
collaborative  data  collection  and  analysis. 

These  and  similar  survey  techniques  are  summarized  in  Appendix  C,  Table  C-4. 

Forecasting  Techniques 

Techniques  for  forecasting  technological  trends  often  play  an  important  part  in 
research  planning,  especially  in  situations  where  incremental  or  leap-ahead  technical 
advancements  can  have  a  major  impact  on  product  perfonnance.  However,  forecasting 
technology  trends  under  these  conditions  is  especially  difficult. 

Some  individuals  like  to  draw  a  distinction  between  forecasting  and  foreseeing 
future  developments.  However,  both  are  an  attempt  to  predict  what  will  happen  in  an 
uncertain  world.  Perhaps  a  more  useful  distinction  is  between  extrapolative  techniques, 
which  are  based  on  projections  derived  from  past  data  and  trends,  and  normative 
forecasting  techniques,  which  rely  on  a  more  subjective  analysis  of  possible  future  events 
and  technology  developments,  where  there  is  little  hard  data  available  on  which  to  base 
decisions.6  Thus,  extrapolative  forecasting  techniques  tend  to  be  quantitative  and  build 
on  actual  data  or  similar  historical  models,  whereas  normative  methods  are  typically 


6 


Brian  Twiss,  Forecasting  for  Technologists  and  Engineers:  A  Practical  Guide  for  Better  Decisions 
(London:  Peter  Perigrinus,  1992). 
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speculative  and  qualitative  and  are  based  on  expert  opinion  and  insight  on  many  technical 
and  nontechnical  factors.  Although  both  approaches  are  important  in  early  stages  of 
R&D,  nonnative  forecasting  methods  are  particularly  useful  in  predicting  the  timing  and 
impact  of  technical  developments  that  have  not  occurred  yet.  Nonnative  techniques  are 
also  used  to  determine  the  feasibility  of  achieving  a  given  technical  or  systems  capability 
within  a  required  time  frame  and  cost  and  thus  are  useful  for  assessment  as  well. 

A  large  number  of  technology  forecasting  techniques  have  been  developed  and 
applied  for  research  planning  purposes,  with  widely  varying  degrees  of  success.  These 
include  methods  such  as: 

•  Trend  extrapolation — Projecting  future  directions  based  on  trends  derived 
from  historical  data. 

•  Pattern  matching  techniques — These  include  the  analysis  of  patents  and 
scientific  papers  in  particular  technical  areas. 

•  The  use  of forecasting  “Jaws”  and  models — These  include  the  use  of  Moore’s 
law  to  predict  the  exponential  growth  in  the  number  of  transistors  per 
integrated  circuit. 

•  Scenario  analysis — Scenario  analysis  results  in  forecasts  that  explicitly 
recognize  that  future  events  are  not  detenninistic  by  examining  the  impact  of 
several  feasible  alternatives.  This  technique  also  helps  decision-makers 
become  more  sensitive  to  signals  of  impending  change. 

•  Expert  opinion — Expert  opinion,  especially  based  on  consensus-building 
approaches,  is  a  widely  used  approach  to  technical  forecasting.  A  good 
example  is  the  Delphi  method,  a  technique  that  uses  iterative  rounds  of  polling 
across  a  group  of  experts  to  arrive  at  a  forecast  of  the  most  probable  outcome 
for  some  future  state. 

These  methods  and  others  are  summarized  in  Appendix  C. 

Other  Tools  and  Techniques 

In  addition  to  the  methods  mentioned  above,  other  types  of  tools  and  techniques 
can  be  used  to  support  research  activities: 

Groupware  for  project  planning  and  management — Software  to  enable 
collaboration  among  research  managers  and  other  participants  in  defining  and 
managing  research  projects  is  becoming  increasingly  popular.  Such  systems 
help  to  automate  the  following  functions:  (1)  capturing  ideas,  (2)  transcribing 
ideas  into  electronic  format,  (3)  organizing  data  and  relationships,  (4)  creating 
action  plans,  and  (5)  distributing  information  electronically.  A  system  of  this 
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type  to  support  MDA/AS’s  requirements  would  most  likely  need  to  be  custom 
designed. 

Environmental  scanning  methods — Keeping  abreast  of  technological  and 
other  developments  that  would  affect  a  research  program  is  a  major  challenge 
for  research  organizations.  Although  environmental  scanning  is  often  viewed 
as  an  integral  part  of  the  strategic  planning  process,  relatively  few  tools  are 
available  to  perform  this  function.  To  overcome  this  problem,  some 
organizations  have  created  dedicated  capabilities  to  perform  the  scanning 
function.  These  include  specific  staff  responsible  for  tracking  and  reporting 
developments  and  trends,  the  use  of  internal  newsletters  to  communicate 
important  infonnation,  subscription  to  various  technical  intelligence  services, 
and  commissioning  periodic  scanning  studies,  among  others. 

Knowledge  management — This  tool  has  not  lived  up  to  its  much-publicized 
potential,  thus  resulting  in  considerable  skepticism  among  research  managers. 
However,  the  basic  premise  of  knowledge  management — namely,  the  need  for 
an  organization  to  more  effectively  utilize  its  knowledge — is  still 
fundamentally  sound.  To  this  end,  research  organizations  are  developing 
tools,  such  as  online  information  repositories,  to  catalog  and  make  available 
relevant  information  to  research  managers  and  other  users  within  the 
organization. 

Mind  mapping — Mind  mapping  is  one  of  several  methods  of  brainstorming, 
planning,  meeting  facilitation,  and  implementing  action  plans  that  rely  on 
conceptual  mapping  techniques  to  capture  and  communicate  concepts  and 
increase  team  collaboration  and  productivity.  Usually  computer-based,  mind 
maps  automate  the  functions  of  (1)  capturing  ideas,  (2)  transcribing  ideas  into 
electronic  format,  (3)  organizing  data  and  relationships,  (4)  creating  action 
plans,  and  (5)  distributing  information  electronically. 

Staffing  experiments — There  has  been  a  growing  awareness  in  recent  years 
that  there  may  be  a  correlation  between  an  individual’s  personality  type  and 
success  as  a  technological  innovator.  Thus,  some  research  organizations  are 
beginning  to  use  personality  type  indicators  (e.g.,  the  Myers-Briggs  Type 
Indicator)  as  one  criterion  when  selecting  research  managers.  This  approach, 
if  it  continues  to  prove  correct,  raises  some  interesting  possibilities  in  helping 
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managers  to  ensure  that  individuals  are  assigned  to  the  right  types  of  jobs  to 
optimize  research  effectiveness. 

These  and  other  tools  are  summarized  in  Appendix  C. 

As  mentioned  earlier,  many  of  the  tools  and  techniques  apply  to  more  than  one 
S&T  planning  and  management  function,  as  illustrated  in  Table  V-l. 

Table  V-1.  Examples  of  Tools  and  S&T  Planning  and  Management  Functions 


To  illustrate  how  these  techniques  could  be  applied  to  MDA’s  S&T  planning  and 
management,  three  examples  have  been  selected  for  further  elaboration:  early  stage-gate 
processes,  scoring  models  for  project  evaluation  and  selection,  and  simulation  and 
modeling.  Stage-gate  and  scoring  models  are  described  in  the  remainder  of  this  chapter. 
The  next  chapter  delves  into  simulation  and  modeling. 


MDA/AS  STAGE-GATE  PROCESS 

As  discussed  previously,  a  stage-gate  process  is  a  structured  framework  for 
managing  R&D  projects,  where  the  stages  represent  the  phases  or  steps  the  project  must 
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progress  through,  and  the  gates  refer  to  review  points  or  intermediate  milestones  where 
the  progress  and  future  direction  of  the  project  are  reviewed  against  a  set  of  previously 
defined  criteria.  Such  processes  are  used  by  the  majority  of  R&D  organizations  to  (1) 
guide  decisions  on  which  project  to  fund;  (2)  align  projects  with  R&D  strategies  and 
organizational  objectives;  (3)  provide  guidance  on  project  definition,  including  scope, 
desired  outputs,  integration,  and  transition  of  results;  and  (4)  review  projects  to  ensure 
progress,  programmatic  fit,  and  priority.7 

A  stage-gate  process  of  this  type  can  be  used  by  MDA/AS  to  plan  and  manage 
early-stage  S&T  projects.  Such  a  process  would  begin  with  idea  generation  and  end  at  the 
point  when  the  project  is  either  transitioned  to  another  MDA  program  element  for  further 
development  or  is  terminated.  The  process  would  be  designed  to  feed  new  component 
technologies  and  systems  concepts  into  MDA’s  overall  RDT&E  pipeline  and  provide  the 
necessary  internal  and  external  linkages  and  integration  to  ensure  success. 

Some  users  have  observed  that  a  traditional  stage-gate  process  is  best  suited  for 
managing  well-defined  product  development  efforts  and  that  it  does  not  handle  the  early 
“fuzzy  front  end”  of  R&D  very  well.  However,  this  criticism  has  been  successfully 
addressed  by  a  number  of  organizations  that  have  modified  the  process  to  encourage  the 
necessary  flexibility  in  the  early  stages  and  still  maintain  a  structured  management 
approach.  One  variation  of  this  approach  is  sometimes  referred  to  as  the  “technology 
stage-gate  method,”  which  has  been  implemented  by  several  firms,  including  DuPont  as 
its  APEX  process,  as  described  in  the  Phase  I  report. 

With  these  comments  in  mind,  an  early  stage-gate  process  along  the  lines 
discussed  below  is  recommended  for  MDA/AS.  This  process,  tentatively  referred  to  as 
the  “MDA  Innovation  Process,”  or  MIP,  is  illustrated  in  Figure  V-2.  This  is  an  initial 
framing  of  the  process,  and  will  need  to  be  refined  in  detail  in  cooperation  with  MDA/AS 
personnel. 


7  Abbie  Griffin,  “Product  Development  Management  Association  Research  on  New  Product 
Development  Practices:  Updating  Trends  and  Benchmarking  Best  Practices,”  Journal  of  Product 
Innovation  Management,  Vol  14,  1997,  pp.  429-458. 
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Figure  V-2.  MDA  Innovation  Process 

The  proposed  MIP  is  a  four-stage  process  that  begins  with  concept  definition  and 
ends  with  either  the  transition  of  the  concept  to  another  MDA  organization  for  further 
development,  or  termination,  which  can  take  place  at  any  of  the  review  gates. 

Before  describing  the  MIP  in  more  detail,  the  reader  should  note  a  few  important 
points.  First,  although  the  MIP  may  look  like  a  uniform  process,  in  reality  it  is  very 
flexible.  It  is  applied  according  of  the  needs  of  the  specific  project  being  reviewed  and 
the  type  of  concept  being  developed.  In  MDA/AS’s  case,  the  majority  of  projects  will  be 
one  of  the  following  types: 

•  Platform  (or  flagship)  projects — These  are  projects  that  would  radically 
change  the  way  missile  defense  is  performed. 

•  Leap-ahead  projects — These  are  projects  that  represent  a  major  advancement 
in  an  underlying  technology,  component,  or  subsystem. 

•  Strategic  technology  projects — These  are  projects  that  represent  emerging 
technologies  that  have  the  potential  to  dramatically  change  components, 
subsystems  or  systems  but  where  the  application  has  not  yet  been  well  defined. 

Because  of  the  significant  differences  between  these  types  of  projects,  the  tasks 
that  would  be  undertaken  in  each  MIP  stage  could  vary  considerably.  The  issues  that 
would  be  examined  during  each  MIP  gate  would  similarly  differ  depending  on  the  type  of 
project  being  reviewed.  Thus,  the  stages  and  gates  for  any  particular  projects  would  be 
custom  tailored  to  the  needs  of  the  concept.  And  it  is  likely  that  MDA/AS  will  undertake 
some  projects  that  would  not  be  managed  with  the  MIP.  These  could  range  from  broad 
investigations  and  planning  studies  to  fast-track  activities  and  more  near-term  problem 
solving. 

The  second  point  is  that  the  process  is  dynamic.  Since  the  stage-gate  process 
serves  as  the  wellspring  of  advanced  concepts  for  MDA/AS,  it  must  accomplish  the 
following  objectives: 
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•  Refining  the  concept — Especially  in  the  early  phases  of  the  process,  refining 
(or  tuning)  the  concept  is  not  an  easy  matter.  For  example,  going  from  the 
concept  of  using  nanotechnology  to  a  specific,  implementable  application 
would  require  considerable  effort  to  sort  out  options  to  ensure  the  right  target 
is  selected. 

•  Determining  if  the  concept  is  actually  possible — In  the  broadest  sense, 
determining  whether  a  concept  has  any  chance  of  really  working  is  not 
straightforward.  A  good  example  here  is  the  case  of  inertial  guidance 
technology.  Early  on,  many  experts  argued  that  it  simply  was  not  possible;  this 
hurdle  had  to  be  overcome  to  proceed  with  development  of  the  concept. 

•  Determining  if  the  concept  is  practical — Many  concepts  may  be  technically 
feasible  but  totally  impractical.  Sometimes  the  introduction  of  a  new 
technology  causes  other  unforeseen  difficulties  that  need  to  be  addressed. 
Again,  inertial  guidance  is  a  good  example.  It  took  many  years  to  achieve  the 
100-fold  or  more  improvements  in  component  technology  needed  to  achieve 
practical  performance  goals. 

•  Determining  if  the  concept  is  desirable — An  innovation  may  be  possible  and 
practical  but  still  not  desirable  for  a  variety  of  economic,  political,  and  social 
reasons.  Similarly,  an  innovation  may  languish  because  it  is  an  unwanted 
“orphan.”  If  such  cases,  the  challenge  would  be  to  implement  a  strategy  that 
will  convert  an  orphan  into  a  prize. 

•  Positioning  the  technology  for  the  next  stage  of  development  or  insertion — The 
technology  stage-gate  process  feeds  into  a  downstream  mechanism  for  further 
development.  Consequently,  the  process  must  lay  the  groundwork  for  this 
transition  to  make  it  as  smooth  as  possible.  This  involves  engaging  the 
necessary  stakeholders  and  generating  the  information  required  to  facilitate  the 
transition.  It  also  implies  bringing  the  technology  to  a  suitable  readiness  level 
for  transition. 

Although  these  objectives  may  appear  linear  (or  sequential),  they  are,  in  fact, 
ongoing  and  take  place  in  parallel.  Depending  on  where  the  concept  is  in  the  stage-gate 
pipeline,  however,  each  objective  has  a  different  level  of  emphasis,  as  illustrated  in 
Figure  V-3.  For  example,  the  issue  of  refining  the  concept  to  be  sure  the  right  idea  is 
being  pursued  is  an  ongoing  challenge.  It  is  especially  important  in  the  early  stages,  but 
the  concept  is  still  fair  game  for  modification  until  the  innovation  emerges  from  the 
pipeline. 
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Figure  V-3.  MDA/AS  Innovation  Process  Stages  and  Objectives 


Even  the  idea  of  concept  definition  is  not  linear.  As  shown  in  figure  V-3,  it  may 
require  many  iterations  and  morphing  to  produce  a  concept  that  is  suitable  for  entry  into 
the  stage-gate  pipeline.  This  initial  phase  is  sometimes  referred  to  as  the  “fuzzy  front 
end”  of  stage-gate  processes.  Thus,  Figures  V-3  and  V-4  show  that  the  proposed 
innovation  process  is  far  from  linear,  though  the  pipeline  itself  is  a  set  of  sequential 
phases. 
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Figure  V-4.  “Fuzzy  Front  End”  of  Stage-Gate  Process 
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The  message  here  is  that  the  innovation  process,  especially  in  its  early  stages,  is 
inherently  difficult  to  predict  because  of  the  dynamic  nature  of  the  issues  and  risks,  and 
each  project  must  be  handled  on  a  case-by-case  basis  as  it  evolves.  And  as  mentioned 
earlier,  an  ability  to  cope  with  and  even  thrive  in  this  type  of  fluid,  vibrant, 
entrepreneurial  environment  must  be  reflected  in  the  staff  assigned  to  manage  and 
participate  in  the  process. 

The  third  point  the  reader  should  note  is  that  there  will  be  significant  effort 
required  of  MDA/AS  both  before  and  after  the  MIP.  For  example,  before  entry  (the 
“discovery”  phase),  a  number  of  important  activities  would  be  performed,  including: 

•  Information  gathering. 

•  Needs  and  technology  analyses. 

•  Opportunity  identification  and  idea  generation.  (These  ideas  and  project 
suggestions  may  come  from  many  sources,  including  Broad  Area 
Announcements,  Small  Business  Innovation  Research  programs,  and  other 
solicitations;  unsolicited  proposals  and  external  project  suggestions;  hunters 
and  gatherers  and  other  internal  MDA  sources;  Strategic  Technology  Steering 
Committees;  and  various  workshops  and  studies.) 

•  Strategic  and  tactical  alignment  of  ideas  to  conform  to  MDA’s  needs  and  S&T 
strategy. 

•  Assignment  of  an  appropriate  project  leader. 

•  Initial  definition  of  the  concept  and  MIP  Stage  0  project. 

Therefore,  even  before  entering  the  MIP,  considerable  work  would  have  been 
accomplished,  and  information  to  be  reviewed  at  Gate  0  (the  on  ramp  to  the  MIP)  would, 
at  minimum,  include 

•  Initial  concept  and  project  definition: 

o  Background  and  description  of  the  technology, 
o  Potential  MDA  applications  and  benefits, 
o  Stage  0  plan  and  resource  requirements. 

•  Identification  of  the  designated  project  leader. 

Similarly,  MDA/AS ’s  work  would  not  stop  at  the  end  of  the  MIP.  Depending  on 
the  needs  of  the  specific  concept,  post-MIP  activities  include,  at  minimum,  transition 
support  to  the  receiving  MDA  organizational  element  and  follow-up  data  collection  to 
track  implementation  issues  and  impact.  In  some  cases,  post-MIP  activities  may  also 
include  transfer  of  personnel  to  assist  in  the  transition. 


V-21 


The  last  important  point  the  reader  should  note  is  that  the  MIP  represents  a 
funnel-shaped  pipeline  in  which  the  number  and  dollar  of  value  of  projects  in  each  stage 
varies  inversely.  Thus,  a  relatively  large  number  of  projects  may  be  underway  in  MIP 
Stage  0,  but  at  each  succeeding  gate,  many  of  these  projects  are  weeded  out,  resulting  in 
a  small  number  of  concepts  that  are  successfully  transitioned  to  other  MDA 
organizational  elements.  Similarly,  as  a  concept  progress  down  the  pipeline,  the  cost  of 
the  next  next-phase  project  increases,  sometimes  dramatically.  This  funneling  affect  is 
illustrated  in  Table  V-2.  For  even  the  best  managed  R&D  organizations,  the  fallout  rate 
from  concept  definition  to  full  implementation  is  about  six  or  seven  to  one.8  Thus,  under 
the  best  of  circumstances,  it  takes  starting  at  least  six  concepts  in  the  funnel  to  yield  one 
successful  implementation.  Moreover,  the  managing  organization  must  be  disciplined  and 
focused  to  move  forward  with  those  projects  that  have  the  highest  relevancy  to  the 
organization  and  to  terminate  or  rework  those  concepts  that  are  of  lesser  priority. 


Table  V-2.  Relative  Number  of  Projects  and  Project  Values  by 
MDA  Innovation  Process  Stage* 


MIP  Stage 

Number  of 
Projects 

Avg.  Project 
Value  ($K) 

Value  by 
Type  ($K) 

%  of  Total 
Funds 

0  -  Concept  definition 

10 

500 

5,000 

5% 

1  -  Explore  key  issues 

5 

5,000 

25,000 

25% 

2  -  Concept-level  experimentation 

2 

10,000 

20,000 

20% 

3  -  Confirmation  and  transition  plan 

1 

50,000 

50,000 

50% 

Totals 

18 

5,600 

100,000 

100% 

*  The  values  used  in  Table  2  are  only  for  the  purpose  of  illustrating  the  relationship  between 
project  stages  and  could  vary  considerably  based  on  actual  circumstances. 

We  discuss  each  MIP  stage  in  turn. 


Stage  0  (Concept  definition) — As  the  name  implies,  the  purpose  of  Stage  1  is 
to  further  define  and  refine  the  concept  that  will  be  pursued  in  the  MIP. 
Typical  tasks  that  would  be  undertaken  in  this  stage  include  the  following: 

a.  Form  a  multidisciplinary  project  team.  The  size  and  composition  of  the 
team  will  obviously  vary  according  to  the  needs  of  the  project,  and  the 
team  can  be  expanded  as  the  concept  progresses  through  the  MIP. 

b.  Identify  ongoing  research,  key  researchers,  and  research  needs. 

c.  Study  MDA  uses,  benefits,  and  important  application  issues. 


8  Ibid. 
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d.  Determine  key  barriers,  research  needs,  and  risks  that  need  addressing. 

e.  If  appropriate,  explore  the  possibility  of  finding  other  research 
collaborators  within  and  outside  DoD  to  leverage  S&T  resources. 

f.  Develop  a  detailed  Stage  1  plan,  success  criteria,  and  resource 
requirements. 

g.  Develop  preliminary  plans  for  Stages  2  and  3  and  estimate  resource  needs. 

h.  Review  Stage  0  findings  with  a  Strategic  Technology  Steering  Group,  if 
appropriate 

i.  Develop  a  team  recommendation  regarding  “go”  or  “no-go”  for  Stage  1 . 

Stage  1  (Exploration  of  key  issues) — During  this  phase,  the  project  will 
focus  on  resoling  the  key  issues  identified  during  Stage  0.  Typical  tasks 
during  Stage  1  include  the  following: 

o  Expand  or  modify  of  the  project  team,  if  needed.  This  may  mean  including 
individuals  from  other  MDA  organizational  elements  that  either  provide 
specialized  expertise  or  would  be  affected  by  downstream  transition  of  the 
technology. 

o  Investigate  key  technical  issues  regarding  concept  feasibility.  Since  MDA 
does  not  conduct  research  internally,  this  effort  would  be  performed  by 
contractors  or  other  government  organizations  under  the  management  of 
the  project  team.  Thus,  making  contractual  arrangements  may  be  a 
significant  activity  during  this  stage. 

o  Update  MDA  applications  information,  associated  benefits,  and  technical 
and  operational  issues  for  the  new  technology, 
o  Review  findings  with  appropriate  stakeholders,  such  as  other  MDA 
organizational  elements. 

o  Develop  a  detailed  Stage  2  plan,  including  success  criteria  and  resource 
requirements. 

o  Update  the  preliminary  plan  for  Stage  3  and  estimated  resource  needs, 
o  Prepare  a  Stage  1  report. 

o  Review  findings  with  a  Strategic  Technology  Steering  Group,  if 
appropriate. 

o  Establish  a  team  recommendation  regarding  “go”  or  “no-go”  for  Stage  2. 
Stage  2  (Concept-level  experimentation) — During  this  stage,  the  innovation 
will  be  investigated  at  the  concept  level  to  evaluate  overall  feasibility.  This 
might  be  similar  to  constructing  and  testing  a  concept-level  “breadboard” 
prototype  to  detennine  initial  feasibility  at  this  level,  as  opposed  to  creating  a 
more  refined  “brass  board”  instantiation  of  the  technology.  Typical  activities 
that  would  be  undertaken  during  MIP  Stage  2  include  the  following: 
o  Further  expansion  or  modification  of  the  project  team. 
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o  Investigation  of  key  technical  issues  regarding  concept-level  feasibility. 
Like  the  previous  stage,  this  effort  would  be  performed  by  contractors  or 
other  government  organizations,  under  the  management  of  the  project 
team.  Thus,  contracting  details  may  be  a  significant  activity  during  this 
stage. 

o  Updating  MDA  applications  information,  associated  benefits,  and 
technical  and  operational  issues  for  the  new  technology, 
o  Reviewing  findings  with  appropriate  stakeholders. 

o  Development  of  an  initial  transition  plan  that  would  address  how  the 
technology  would  be  transitioned  within  MDA  at  the  end  of  the  MIP. 
o  Updating  the  Stage  3  plan,  success  criteria  and  resource  requirements 
o  Preparation  of  a  Stage  2  report. 

o  Reviewing  findings  with  a  Strategic  Technology  Steering  Group,  if 
appropriate. 

o  Establishing  a  team  recommendation  on  “go”  or  “no-go”  for  Stage  3. 

Stage  3  (Concept-level  confirmation  and  transition  planning) — During 
this  stage,  the  final  step  in  the  MIP,  the  concept-level  feasibility  for  the 
innovation  is  confirmed,  and  all  details  concerning  how  the  technology  will  be 
transitioned  to  other  elements  within  MDA  are  resolved.  Activities  during 
Stage  3  would  include  the  following: 

o  Further  expansion  or  modification  of  the  project  team,  if  needed, 
o  Additional  concept-level  demonstrations  needed  to  confirm  the  feasibility 
of  the  innovation.  As  in  the  previous  stage,  this  effort  would  be  performed 
by  contractors  or  other  government  organizations  under  the  management 
of  the  project  team.  Thus,  contracting  details  may  be  a  significant  activity 
during  this  stage. 

o  Finalizing  MDA  applications  information,  associated  benefits,  and 
technical  and  operational  issues  for  the  new  technology, 
o  Reviewing  findings  with  appropriate  stakeholders, 
o  Finalizing  the  technology  transition  plan, 
o  Preparation  of  a  Stage  3  report. 

o  Reviewing  findings  with  a  Strategic  Technology  Steering  Group,  if 
appropriate. 

o  Establishing  a  team  recommendation  on  “go”  or  “no-go”  for  transition. 

At  this  point,  the  innovation  will  have  completed  its  running  of  the  MIP  gantlet 
and  would  then,  with  appropriate  Gate  4  approval,  move  into  the  transition  phase  and 
integrate  with  other  MDA  programs.  As  mentioned  previously,  even  though  the  MIP  has 
been  completed  for  this  technology,  MDA/AS  would  still  perform  some  post-MIP 
activities  to  support  the  transition  and  track  the  subsequent  success  (or  failure)  of  the 
concept. 
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It  should  again  be  emphasized  that,  although  the  stages  are  conceptually  the  same, 
the  actual  work  to  be  performed  during  each  stage  may  vary  considerably  for  any 
particular  project,  depending  on  the  type  of  innovation,  the  level  of  its  application 
(e.g.,  component,  subsystem  or  system),  and  the  unique  circumstances  of  the  technology 
being  developed.  One  of  the  keys  to  using  this  type  of  process  is  the  customization  of  the 
work  activities,  project  team,  and  success  criteria,  for  the  innovation  being  pursued. 
During  the  early  stages,  the  emphasis  is  on  technical  feasibility;  in  later  phases,  the  issues 
of  specific  MDA  applications,  affordability,  overall  desirability,  and  transitioning 
become  increasingly  important.  Similarly,  the  MIP  funnel  focuses  larger  resources  on 
fewer  projects  and  involves  wider  ranges  of  participants  and  stakeholders  as  the 
innovations  flow  through  the  process. 

One  of  the  key  distinctions  between  a  technology  stage-gate  process,  such  as 
MIP,  and  a  traditional  stage-gate  process  is  that  the  development  of  technologies  is 
fundamentally  different  than  development  of  products.  Product  development  using  a 
traditional  stage-gate  method  assumes  that  the  underlying  technologies  are  mature  and 
available.  Thus,  the  emphasis  in  traditional  stage-gate  processes  is  on  managing 
schedules  and  budgets.  Technology  development,  on  the  other  hand,  is  in  inherently 
unpredictable,  especially  in  the  early  stages,  where  capabilities  are  explored,  feasibility  is 
evaluated,  breakthroughs  occur,  and  new  solution  paths  are  uncovered  and  pursued. 
Consequently,  technology  stage-gate  processes  must  be  more  flexible  and,  in  the  earlier 
stages,  focus  on  progress  toward  meeting  essential  technology  performance  goals.  These 
differences  between  traditional  and  technology  stage-gate  processes  are  also  reflected  in 
the  review  gates  that  are  used  to  detennine  the  future  of  the  project  as  it  flows  through 
the  funnel. 

The  MDA  Innovation  Process  employs  five  gates: 

MIP  Gate  0  (On  ramp) — This  initial  gate  detennines  whether  an  innovation 
is  ready  to  enter  the  MIP  funnel.  The  infonnation  to  be  reviewed  by  the 
gatekeepers  (discussed  later)  will  primarily  be  contained  in  an  initial  concept 
definition  document.  As  a  minimum,  this  document  will  address  (1)  a 
description  of  the  technology  and  why  it  is  important,  (2)  potential  MDA 
applications  and  benefits,  (3)  required  advancements  or  challenges  that  need 
to  be  resolved  in  order  to  use  the  technology,  and  (4)  a  Stage  0  plan  and 
resource  requirements.  Depending  on  the  project,  other  information  may  also 
be  included  in  the  initial  concept  definition  document. 
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During  the  Gate  0  review,  the  decision-makers  will  determine  the  fate  of  the 
project  primarily  based  on  (1)  the  potential  for  significant  MDA  applications, 
(2)  how  well  the  project  fits  with  MDA  S&T  strategies  and  portfolio  balance, 
and  (3)  the  adequacy  of  the  Stage  0  plans.  The  gatekeepers  may  decide  to 
approve  entry  into  Stage  0,  terminate  the  project,  or  send  it  back  to  the 
discovery  stage  for  rework. 

MIP  Gate  1  (Entry  to  Stage  1,  exploration  of  key  issues) — At  this  stage  in 
the  MIP,  the  gatekeepers  will  have  a  much  better  definition  of  the  potential  of 
the  technology,  key  issues  that  need  to  be  explored,  and  initial  resource 
requirements.  The  outputs  from  Stage  0  that  feed  into  Gate  1  include  the  Stage 
0  report;  results  of  any  Strategic  Technology  Steering  Group  reviews;  a 
detailed  Stage  1  plan,  success  criteria,  and  resource  requirements;  initial  Stage 
2  and  3  plans  and  resource  requirements;  and  a  project  team  “go”  or  “no-go” 
recommendation.  They  will  use  this  infonnation  to  decide  if  the  project 
should  move  to  the  next  phase  where  the  research  truly  begins. 

Criteria  that  will  be  used  to  make  the  Gate  1  decision  include  (1)  the  potential 
for  significant  MDA  applications  and  long-term  impact;  (2)  the  fit  within 
MDA’s  S&T  strategies  and  portfolio  balance;  (3)  adequacy  of  identification 
of  key  issues,  challenges,  and  risks;  (4)  reasonableness  of  proposed  Stage  1 
plan,  success  criteria,  and  resources;  and  (5)  downstream  Stage  2  and  3  plans 
and  resource  implications. 

It  is  at  this  gate  that  many  of  the  concepts  will  be  screened  out.  In  the 
examples  given  earlier  in  Table  V-2,  as  many  as  half  or  more  of  the  concepts 
would  either  be  terminated,  postponed,  or  sent  back  to  Stage  0  for  rework.  It 
is  this  weeding  out  of  lower  priority  projects  that  many  organizations  find 
difficult.  However,  this  screening  function  is  essential  to  maintaining  a  viable, 
responsive,  and  effective  S&T  program.  This  issue  is  discussed  later. 

MIP  Gate  2  (Entry  into  Stage  2,  initial  concept-level  investigations) — 

During  this  review,  the  gatekeepers  will  decide  if  the  key  issues  affecting  the 
feasibility  of  the  innovation  have  been  adequately  addressed  and  if  enough  is 
known  about  the  technology  to  begin  concept-level  investigation.  The  outputs 
from  Stage  1  that  feed  into  Gate  2  include  (1)  resolution  of — or  at  least 
significant  progress  on — the  key  issues;  (2)  the  Stage  1  report;  (3)  results  of 
any  Strategic  Technology  Steering  Group  reviews;  (4)  a  detailed  Stage  2  plan, 
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success  criteria,  and  resource  requirements;  (5)  an  updated  preliminary  Stage 
3  plan  and  resource  requirements;  and  (6)  a  project  team  “go”  or  “no-go” 
recommendation  for  entry  into  Stage  2. 

As  with  the  earlier  gates,  the  decision-makers  will  decide  to  move  the  concept 
to  the  next  stage,  tenninate  it,  or  continue  it  in  the  earlier  stage  for  more 
investigation.  Again,  for  the  MIP  to  work  effectively,  only  a  limited  number 
of  high-priority  concepts  will  be  allowed  to  move  forward  in  the  pipeline.  The 
criteria  that  will  be  used  to  make  this  decision  include  (1)  potential  for  MDA 
applications  and  long-term  impact;  (2)  continuing  fit  with  MDA’s  S&T 
strategies  and  portfolio  balance;  (3)  resolution  of  key  issues,  challenges,  and 
risks  (Is  the  technology  is  really  understood  and  suitable  for  the  proposed 
applications?);  (4)  adequacy  of  the  proposed  Stage  2  plan,  success  criteria, 
and  resources;  (5)  downstream  Stage  3  plan  and  resource  implications;  and 
(6)  involvement  of  other  MDA  elements  and  DoD  organizations. 

MIP  Gate  3  (Entry  into  Stage  3,  concept-level  confirmation  and 
transition  plan) — At  this  gate,  the  decision-makers  will  determine  if  the 
concept  moves  to  the  final,  most  expensive,  and  perhaps  most  critical  MIP 
stage.  At  this  stage  it  will  complete  the  groundwork  for  transitioning  the 
concept  to  other  MDA  organizational  elements  and  subsequent  development 
and  deployment  of  the  technology.  Outputs  from  Stage  2  to  be  reviewed  by 
the  gatekeepers  include  (1)  an  initial  assessment  of  the  concept-level 
feasibility  of  the  technology;  (2)  an  initial  transition  plan  for  the  technology; 
(3)  a  Stage  2  report;  (4)  results  of  any  Strategic  Technology  Steering  Group 
reviews;  (5)  a  detailed  Stage  3  plan,  success  criteria,  and  resource 
requirements;  and  (6)  a  project  team  “go”  or  “no-go”  recommendation 
regarding  moving  forward  to  Stage  3. 

As  with  the  previous  gate,  the  criteria  that  will  be  used  to  make  this  decision 
include  (1)  reaffirming  the  potential  for  MDA  applications  and  long-tenn 
impact;  (2)  continuing  fit  with  MDA’s  S&T  strategies  and  portfolio  balance; 
(3)  resolving  all  key  issues,  challenges,  and  risks;  (4)  determining  the 
adequacy  of  the  proposed  Stage  3  plan,  success  criteria,  and  resources;  (5) 
determining  the  adequacy  of  the  initial  transition  plan;  and  (6)  involving  other 
MDA  elements  and  DoD  organizations.  The  primary  focus  of  the  decision¬ 
makers  will  be  on  whether  Stage  3  will  resolve  all  the  issues  that  will  enable  a 
successful  transition  of  the  concept  after  it  leaves  the  MIP. 
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MIP  Gate  4  (Off  ramp,  the  final  MIP  gate)  — This  gate  represents  both  the 
final  MIP  gate  and  the  beginning  of  another  process,  one  that  will  eventually 
lead  to  development  and  deployment  of  the  technology  by  other  MDA 
organizational  elements.  Thus  the  gatekeepers  in  this  instance  will  be  not  only 
the  MIP  decision-makers  but  also  the  senior  managers  that  will  determine  the 
fate  of  the  technology  after  it  leaves  the  MIP.  Outputs  from  Stage  3  that  will 
be  reviewed  during  this  gate  include  (1)  the  final  assessment  of  the  concept- 
level  feasibility  of  the  technology,  (2)  a  finalized  transition  plan  for  the 
technology,  (3)  a  Stage  3  report,  (4)  results  of  any  Strategic  Technology 
Steering  Group  reviews,  and  a  (5)  project  team  “go”  or  “no-go” 
recommendation  regarding  transition. 

The  primary  criteria  that  will  be  used  during  Gate  4  are  (1)  reaffirmation  of 
the  potential  for  MDA  application  and  long-tenn  impact;  (2)  resolution  of  all 
key  issues,  challenges,  and  risks;  and  (3)  transition  readiness. 

The  MIP  gates  are  fairly  straightforward,  but  their  use  implies  several  subtleties 
that  need  to  be  understood.9  First,  because  MIP  is  a  technology  stage-gate  process  rather 
than  a  traditional  stage-gate  mechanism,  the  MIP  gates  are  driven  primarily  by 
technology  advancements,  which  are  inherently  less  predictable  than  traditional  stage- 
gate  phases.  This  means  that  MIP  gates  will  often  be  event  driven  rather  than  schedule 
driven,  or  more  likely,  driven  by  both  events  and  schedules.  Thus,  MIP  gates  may  be  held 
to  coincide  with  achieving  a  particular  successful  perfonnance  level  for  a  technology. 
Similarly,  MIP  gates  may  be  held  when  a  predesignated  showstopping  event  has 
occurred,  such  as  concluding  that  a  particular  technical  capability  is  not  feasible.  These 
event-driven  milestones  for  MIP  gates  would  be  specified  in  the  plans  for  a  particular 
stage.  In  the  event  that  such  milestones  are  not  appropriate  or  achievable,  then 
conventional  schedule-driven  gate  reviews  would  be  held. 

Another  subtlety  concerns  the  makeup  of  the  gatekeepers.  The  gatekeepers  are  the 
group  responsible  for  conducting  the  gate  reviews  and  deciding  whether  a  project  should 
move  to  the  next  stage.  This  committee  also  approves  the  tasks  to  be  accomplished 
during  that  next  phase  and  the  resource  levels,  subject  to  the  nonnal  budgetary 
procedures  for  the  project.  Typically,  this  committee  would  be  chaired  by  the  head  of 


9  Greg  M.  Ajamian,  and  Peter  A.  Koen,  “Technology  Stage-Gate™:  A  Structured  Process  for  Managing 
High-Risk  New  Technology  Projects,”  in  the  PDMA  Tool  book for  New  Product  Development  (Product 
Development  and  Management  Association,  John  Wiley  &  Sons,  2002)  pp.  267-295. 
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MDA/AS  and  would  have  additional  members  from  within  MDA/AS  and  possibly  from 
other  MDA  organizational  elements  (e.g.,  MDA/SE  and  one  or  more  MDA  project 
elements).  In  some  cases,  a  project  may  also  be  reviewed  by  other  committees.  For 
example,  a  project  relating  to  a  key  strategic  technology  may  also  be  reviewed  by  the 
MDA  steering  committee  for  that  technology. 

In  the  early  stages,  the  gatekeepers  would  be  a  limited  number  of  individuals, 
primarily  from  within  MDA/AS,  that  would  decide  the  fate  of  the  projects,  subject  to 
review  by  senior  MDA  management.  This  is  necessary  to  ensure  that  innovative  concepts 
are  not  killed  prematurely.  As  the  concept  passes  through  the  MIP  funnel,  however,  the 
group  of  individuals  serving  on  the  gatekeeping  committee  would  be  expanded  to  include 
other  stakeholders  within  MDA,  depending  on  the  specifics  of  the  concept  being 
investigated.  Obviously,  these  details  of  the  makeup  of  the  gate  keeping  committees  will 
need  to  be  worked  out  by  MDA/AS. 

Because  stage-gate  processes  have  become  widely  used  in  R&D  organizations, 
several  best  practices  have  been  developed  on  how  gatekeeping  should  be  perfonned. 
These  gatekeeper  rules  of  engagement  include  the  following:10 

•  Gatekeepers  must  hold  the  meeting  and  be  there.  Postponed  or  canceled 
meetings  are  not  permitted,  and  those  not  attending  are  considered  to  be  voting 
“yes”  for  the  project. 

•  Gatekeepers  must  have  received  and  read  the  meeting  materials  and  be 
prepared  for  the  meeting.  No  last-minute  reading  is  permitted  at  the  meeting.  If 
there  are  showstoppers,  then  the  meeting  facilitator  is  contacted  in  advance  so 
that  participants  are  not  surprised. 

•  Gatekeepers  cannot  request  information  beyond  that  specified  in  the  stage 
deliverables. 

•  Gatekeepers  must  make  their  decisions  based  on  the  criteria  specified  for  that 
gate.  Each  criterion  must  be  addressed,  and  a  conclusion  reached  by  the  group. 
A  scorecard  will  be  filled  out  by  each  gatekeeper. 

•  Decisions  must  be  based  on  objective  facts  and  criteria,  not  emotions  or  hidden 
agendas.  All  projects  must  be  treated  fairly  and  consistently,  including  uniform 
application  of  gates. 

•  A  decision  must  be  made  at  the  meeting,  and  the  project  team  must  be 
informed  immediately  and  face  to  face. 


10  R.G.  Cooper,  S.J.  Edgett,  and  E.J.  Kleinschmidt,  “Optimizing  the  Stage-Gate  Process:  What  Best 
Practice  Companies  are  Doing — Part  II,”  Research-Technology’  Management,  Vol  45,  No  5,  2002. 
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Sometimes  gatekeeper  meetings  are  difficult  to  complete  because  of  the  busy 
schedules  of  senior  personnel,  travel  pressures  for  geographically  separated  members, 
and  conflicting  workload  priorities.  To  overcome  these  problems,  some  companies  have 
been  experimenting  with  concepts  such  as  virtual  gate  meetings  where  only  the  project 
team  is  physically  at  the  meeting  place.  The  gatekeepers  receive  the  preparatory 
documents  in  advance  and  participate  electronically  (e.g.,  by  video  conference).  They 
also  submit  their  scores  on-line.  The  scores  are  discussed  until  a  consensus  is  reached, 
after  which  the  results  are  discussed  with  the  project  team.  Some  organizations  are  also 
experimenting  with  self-managed  gates,  in  which  the  project  team  also  serves  as  the 
gatekeepers.  However,  this  in  only  used  when  the  risks  are  relatively  low.  A  variation  on 
this  approach  is  to  use  gatekeepers  that  are  not  part  of  the  normal  stage-gate  process, 
thereby  providing  a  type  of  peer  review  of  the  project.  Some  organizations  are  also 
beginning  to  encourage  the  project  teams  to  make  its  own  recommendations  before  the 
gate  meeting.  In  this  way,  the  actual  gatekeepers  are  viewed  as  more  of  a  secondary 
review  panel,  thus  avoiding  the  necessity  of  boring  down  to  investigate  details. 

In  addition  to  gatekeepers,  other  key  roles  (or  actors)  in  the  MIP  include  the 
following: 

Process  owner — This  is  the  individual  responsible  for  defining  and  updating 
the  technology  stage-gate  process,  championing  its  use  within  MDA,  tracking 
the  projects  within  the  process,  and  serving  as  a  process-related  resource  to 
other  participants. 

Project  leader — Every  process  that  enters  the  MIP  pipeline  will  have  an 
assigned  project  leader.  It  is  this  individual’s  responsibility  to  manage  the 
project  as  it  flows  through  the  stages  and  gates.  This  individual  is  accountable 
for  project  deliverables  and  coordinating  and  overseeing  the  activities  of  the 
project  team. 

Project  team — This  is  a  cross-functional  group  that,  along  with  the  project 
leader,  is  responsible  for  overseeing  the  activities  carried  out  during  the 
various  process  stages.  The  membership  of  the  team  will  depend  on  the  nature 
of  the  specific  project  and  may  change  and  expand  as  the  project  progresses 
down  the  pipeline. 

S&T  performer(s) — In  most  cases,  the  projects  will  be  executed  by 
organizations  outside  MDA,  either  contractors  or  other  governmental 
organizations.  Most  likely,  one  or  more  individuals  from  the  performing 
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organization(s)  will  participate  in  at  least  part  of  the  review  gate  activities, 
particularly  regarding  work  that  has  been  accomplished  or  is  planned  for 
subsequent  stages. 

Several  new  product  development  consultants  and  practitioners  have  identified 
important  attributes  or  “best  practices”  that  affect  the  success  of  stage-gate  processes. 
These  keys  to  success  include  the  following:11 

Leadership  support — Like  most  changes  within  organizations,  the 
implementation  of  a  stage-gate  process  does  best  when  it  is  supported  by  top 
management.  In  this  case,  however,  such  support  is  not  only  desirable,  it  is 
critical  if  the  process  is  to  become  an  effective  means  of  generating  essential 
missile  defense  concepts  and  technologies.  As  a  minimum,  top  management 
needs  to  participate  in  the  kick-off  of  the  process  and  be  visible  and 
supportive  in  the  various  review  stages.  Having  a  senior  manager  serve  as  an 
executive  sponsor  and  advocate  for  the  process  is  also  desirable.  And  last  but 
not  least,  having  senior  executives  acknowledge  and  reward  those  individuals 
responsible  for  the  success  of  the  process  is  an  important  aspect  of  any  type  of 
change  management. 

Adequate  resources — Having  available  adequate  resources  for  implementing 
the  process  and  supporting  the  resulting  S&T  projects  is  critical  to  success. 
Without  adequate  resources,  quality  of  execution  and  team  morale  suffer,  time 
delays  occur,  and  only  low-impact  technologies  emerge  from  the  process. 

An  appropriate  process  design — Although  several  standard  stage-gate 
processes  have  been  acquired  from  various  consulting  organizations,  it  is 
important  that  the  process  chosen  for  implementation  be  tailored  specifically 
to  the  needs  and  characteristics  of  the  using  organization.  “Best  practices” 
used  by  other  organizations  are  important,  but  they  first  need  to  be  subjected 
to  a  critical  assessment  to  be  sure  that  they  can  be  properly  used  in  a  particular 


1 1  Barbara  M.  Pitts,  and  L.  Michelle  Jones,  “Successfully  Implementing  the  Stage-Gate™  New  Product 
Development  Process,”  Working  Paper  No.  18,  Product  Development  Institute,  Inc.,  Ancaster, 
Ontario,  Canada,  2003;  R.G.  Cooper,  and  S.J.  Edgett,  “Overcoming  the  Current  Crunch  in  New 
Product  Development  Resources,”  Working  Paper  No.  17,  Product  Development  Institute,  Inc., 
Ancaster,  Ontario,  Canada,  2003;  R.G.  Cooper,  S.J.  Edgett,  and  E.J.  Kleinschmidt,  “Optimizing  the 
Stage-Gate  Process:  What  Best  Practice  Companies  are  Doing — Part  I,”  Research-Technology’ 
Management,  Vol.  45,  No.  3,  2002,  pp.  21-27;  Cooper,  et  al.,  “Optimizing  the  Stage-Gate  Process — 
Part  II;”  and  A.  Khurana,  and  S.R.  Rosenthal,  “Towards  Holistic  ‘Front  Ends’  in  New  Product 
Development,”  Journal  of  Product  Innovation  Management,  Vol.  15,  1998,  pp.  57-74. 
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situation.  Thus,  each  stage-gate  process  must  be  customized  to  meet  the  using 
organization’s  needs  prior  to  implementation.  Areas  that  are  particularly 
troublesome  in  process  design  include  structuring  the  “fuzzy  front  end,” 
establishing  appropriate  cross-functional  teams,  and  creating  and 
implementing  tough  decision  gates.  It  is  also  important  that  the  process  design 
be  holistic,  in  that  it  provides  for  the  integration  of  the  S&T  strategy  with  the 
organization’s  mission  and  product-development  strategies  and  procedures. 

Defined  roles  and  responsibilities — Implementing  a  stage-gate  process 
requires  the  coordinated  efforts  of  many  people,  and  it  is  important  that  these 
roles  and  responsibilities  be  defined  in  advance  to  ensure  effective 
implementation.  Of  particular  importance  are  the  gatekeepers  (i.e.,  decision¬ 
makers);  the  project  leaders  and  teams;  and  the  process  manager,  the 
individual  responsible  for  defining,  championing,  and  implementing  the 
process. 

Implementation  plan  matched  to  organizational  situation — Without  a  clear 
mandate  and  visible  support  from  top  management  for  implementing  a  new 
planning  and  management  process,  it  is  necessary  to  adopt  a  more 
nonthreatening,  “grass  roots”  approach  that  will  build  acceptance  among 
middle  managers,  technical  personnel,  and  other  participants.  Such  an 
implementation  strategy  requires  involving  the  affected  stakeholders,  along 
with  initial  trials  and  successes  that  can  lead  to  successful  institutionalization 
of  the  process.  Thus,  an  effective  implementation  plan  must  be  both 
participatory  and  integrative  to  achieve  sustainable  results.  And  because  each 
organization’s  culture  is  different,  the  implementation  plan  must  also  be 
designed  with  an  eye  toward  lessons  learned  from  other  successful 
implementations  that  have  been  achieved  by  the  organization. 

Effective  communication — A  significant  level  of  effort  is  invested  in 
designing  an  appropriate  process,  and  it  is  important  that  the  insight  gained 
from  this  effort  be  communicated  to  all  the  participants  to  increase  their 
knowledge  and  obtain  as  much  buy-in  as  possible.  Therefore,  a 
communication  plan  should  be  part  of  the  implementation  strategy. 

Focus  and  discipline — Two  of  the  most  common  problems  encountered  with 
new  stage-gate  processes  are  undertaking  too  many  projects  and  the  inability 
to  terminate  marginal  or  underperforming  projects.  There  are  many 
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underlying  reasons  for  these  difficulties,  particularly  regarding  the  inability  to 
kill  projects.  These  reasons  include  pursuit  of  pet  projects  or  those  mandated 
as  “must  do”  by  senior  management  or  external  organizations,  unwillingness 
to  cancel  efforts  that  represent  large  sunk  costs,  and  lack  of  effective  gating 
mechanisms  to  control  what  flows  through  the  S&T  pipeline.  A  number  of 
techniques  can  be  used  to  focus  the  process,  such  as  creating  an  explicit  S&T 
innovation  strategy  to  set  direction  and  guide  activities,  applying  schemes  to 
establish  priorities  within  stages  to  determine  which  projects  are  within 
budget  limits,  and  using  a  portfolio-management  process  to  ensure  balance 
among  competing  resource  demands.  In  the  end,  it  is  far  better  to  adequately 
fund  a  few  good  projects  than  to  attempt  to  pursue  too  many  efforts  that  are 
underfunded  and  understaffed,  and  thus  unlikely  to  succeed. 

Progress  measurement — A  few  carefully  chosen  metrics  should  be  selected  to 
measure  the  progress  made  in  implementing  the  stage-gate  process.  These 
metrics  should  include  tracking  implementation  milestones  against  target 
dates;  the  number  of  projects  in  each  stage;  dwell  times,  attrition  rates  and 
reasons;  progress  toward  meeting  overall  process  goals  (e.g.,  number  of 
projects  by  stage,  funding  levels,  and  number  of  successes);  and  the  impact  of 
success  stories. 

The  above  summarizes  an  approach  for  implementing  the  MIP  technology  stag- 
gate  processes  within  MDA/AS.  Such  a  process  would  provide  several  important 
benefits,  including:  (1)  an  effective  method  for  deciding  which  project  to  fund; 

(2)  aligning  projects  with  MDA  S&T  strategies  and  organizational  objectives; 

(3)  establishing  guidance  on  project  definition,  including  scope,  desired  outputs, 
integration,  and  transition  of  results;  and  (4)  reviewing  projects  to  ensure  progress, 
programmatic  fit,  and  priority.  The  overall  impact  of  the  MIP  would  be  to  streamline 
early-stage  S&T  activities  within  MDA/AS  while  balancing  the  creative  technology  and 
pragmatic  applications  perspectives. 

MDA/AS  SCORING  MODELS  FOR  SETTING  PRIORITIES 

One  of  the  main  challenges  in  planning  an  S&T  program  is  ranking  the  project 
alternatives  to  determine  the  highest  priority  options  to  be  pursued.  Scoring  models 
represent  an  increasingly  used  decision-making  technique  that  can  help  support  this 
ranking  function. 


V-33 


Traditionally,  fonnal  scoring  models  have  not  been  widely  used  to  rank  S&T 

investment  alternatives.  For  example,  here  are  five  common  approaches  for  allocating 

12 

resources  that  do  not  rely  on  fonnal  scoring  models: " 

•  Squeaking  wheel — Resources  are  minimized  or  reduced  in  all  areas,  and 
depending  on  which  area  complains  the  loudest  and  most  insistently,  budgets 
are  restored  until  the  funding  ceiling  is  reached. 

•  Level  funding — Budget  changes  are  minimized,  and  the  status  is  quo 
maintained  to  the  detriment  of  new  initiatives. 

•  Glorious  past — Resources  are  assigned  on  the  basis  of  past  record  of 
achievement. 

•  White  charger — The  best  speaker,  department  with  the  best  presentation,  or 
last  person  to  brief  the  decision-maker  receives  the  resources. 

•  Advisoiy  committee — A  committee  reviews  the  alternatives  and  advises  the 
decision-maker  on  how  to  allocate  the  resources. 

Although  the  above  methods  lack  a  scientific  or  objective  basis,  they  are  widely 
used  in  many  organizations.  Often  such  techniques  fail  to  adequately  take  into 
consideration  the  “bottom  line”  impact  of  the  investments  on  the  organization  and  its 
products.  Consequently,  S&T  investments  tend  to  be  driven  by  short-term  priorities  and 
pressures  and  do  not  address  broader  long-term  needs  and  opportunities  for  higher 
payoffs. 

To  improve  this  situation,  many  organizations  have  turned  to  using  scoring 
models  as  decision-making  tools  for  evaluating  and  ranking  alternative  S&T  investments. 
This  section  of  the  report  addresses  the  types  of  scoring  models  that  could  be  used  by 
MDA/AS  in  planning  and  managing  its  S&T  program. 

Any  decision-making  tool  must  be  developed  to  fit  the  context  in  which  it  will  be 
used.  In  this  case,  the  scoring  model  will  be  employed  to  rank  alternative  S&T  projects, 
especially  in  the  early  phases  of  MDA’s  technology  stage-gate  process  (see  separate 
section  describing  the  MDA  Innovation  Process).  The  score  a  particular  candidate  project 
receives  from  the  model  will  largely  determine  (or  at  least  greatly  affect)  whether  the 
project  is  funded  and  enters  the  S&T  pipeline.  Thus,  use  of  the  scoring  model  would  be  a 


M.J.  Cetron,  “Technology  Forecasting  for  the  Military  Manager,”  in  An  Introduction  to  Technological 
Forecasting,  Edited  by  J.P.  Marino,  Gordon  &  Breach,  London,  1972;  M.R.  Kirby,  and  D.N.  Marvis, 
A  Technique  for  Selecting  Emerging  Technologies  for  a  Fleet  of  Commercial  Aircraft  to  Maximize 
R&D  Investment,  SAE  Paper  2001-01-3018,  2001. 
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key  part  of  the  review  that  takes  place  during  MIP  Gate  0,  the  on-ramp  for  the  stage-gate 
process. 

Although  scoring  models  are  frequently  used  for  selecting  among  project 
alternatives,  they  represent  only  one  of  several  approaches.13  These  techniques  can  be 
grouped  into  four  categories: 

•  Intuition — Purely  intuitive  judgment  is  sometimes  used  for  decision-making, 
particularly  for  selecting  smaller  projects  and  in  emergency  or  time-limited 
situations.  This  approach  is  generally  viewed  as  the  most  unreliable  except  for 
all  but  the  most  routine  decisions.  Unfortunately,  many  individuals — including 
most  managers — have  an  unreasonable  faith  in  their  intuition.  (Although 
intuition  is  not  strictly  speaking  a  scoring  model,  it  is  included  here  for 
comparison  purposes.) 

•  Rules — Heuristic  rules  (e.g.,  to  fund  S&T  projects  that  support  a  major 
organizational  strategy  regardless  of  the  intrinsic  merit  of  the  project)  are  easy 
to  employ  and  are  sometimes  used  to  defend  decisions.  However,  such  rules 
inherently  interject  strong  biases  in  the  decision-making  process. 

•  Decision  weighting  (i.e.,  scoring  models) — This  approach  entails  identifying 
and  weighting  those  factors  that  are  most  important  to  the  decision  and  then 
scoring  the  alternative  projects  in  terms  of  these  factors.  Those  projects 
receiving  the  higher  scores  are  given  higher  rankings. 

•  Value  analysis — Systematic,  detailed  value-analysis  techniques  are  sometimes 
used  to  evaluate  project  alternatives.  Although  such  methods  can  produce  the 
best  decisions,  they  can  be  difficult  to  implement  and,  if  not  carefully 
structured  and  documented,  may  not  provide  transparency  into  the  underlying 
basis  for  the  decisions. 

Table  V-3  gives  a  comparison  of  the  advantages  and  disadvantages  of  each 
technique  category. 


Table  V-3.  Pros  and  Cons  of  Different  Decision-making  Approaches 
(Source:  Modified  from  Russo  and  Schoemaker,  Winning  Decisions) 


Decision  Method 
Intuition 
Rules 
Weighting 
Value  Analysis 


Quality  and 
Reliability 

Low 

Moderate 

High 

Very  High 


Level  of  Effort 
Low 

Low  to  Moderate 
High  to  Low 
Very  High 


Transparency 
Very  Low 
Moderate 
Very  High 
High  to  Low 


13 


J.  Edward  Russo,  and  Paul  J.H.  Schoemaker,  Winning  Decisions:  Getting  it  Right  the  First  Time, 
Doubleday,  2002. 
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In  general,  organizations  should  use  the  decision-making  approach  that  provides 
the  highest  levels  of  quality  and  reliability  consistent  with  the  resources  available  and 
context  of  the  decision.  In  selecting  projects  to  enter  into  an  S&T  pipeline  (or  funnel),  the 
specific  details  of  the  technology  or  innovation  are  usually  not  defined  well  enough  to 
apply  detailed  value-analysis  techniques;  the  necessary  data  usually  do  not  exist,  hence 
the  need  for  the  S&T  project.  Thus,  in  the  early  phases  of  a  technology  stage-gate 
management  process,  weighting  techniques  are  the  most  suitable  decision-making  aid. 
However,  value-analysis  methods  may  be  more  applicable  in  later  phases  of  the  stage- 
gate  process,  when  more  data  become  available. 

A  large  number  of  weighting  techniques  and  scoring  models  have  been  developed 
for  evaluating  decision  alternatives,  including  the  ranking  and  selection  of  S&T  projects. 
One  of  the  most  widely  used  techniques  is  the  Kepner-Tregoe  decision-analysis  method, 
which  was  developed  in  the  late  1950s.14  This  method  is  a  systematic  procedure  for 
making  choices  based  on  the  following  activities: 

1 .  State  the  decision  to  be  made. 

2.  Develop  objectives. 

3.  Classify  objectives  into  MUSTs  and  WANTs. 

4.  Weight  the  WANTs. 

5.  Identify  the  alternatives  (in  this  case,  S&T  projects  or  technology  areas). 

6.  Screen  the  alternatives  through  the  MUSTs. 

7.  Compare  (score)  the  alternatives  against  the  WANTs. 

8.  Identify  adverse  consequences. 

9.  Make  the  best  balanced  choice. 

Although  the  process  seems  detailed,  it  can  be  implemented  quickly,  with 
relatively  little  effort.  The  scoring  model  is  developed  and  applied  in  steps  2,  3,  4,  and  7. 

Applying  the  Kepner-Tregoe  technique  to  the  ranking  of  potential  MDA  S&T 
projects  would  be  fairly  straightforward.  The  objectives  and  associated  weightings  need 
to  be  developed  with  the  full  participation  of  MDA  management,  however. 


14  Charles  H.  Kepner,  and  Benjamin  B.  Tregoe,  The  New  Rational  Manager,  Princeton  Research  Press, 
1997. 
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As  an  example,  the  following  WANTs  objectives  (i.e.,  evaluation  criteria)  and 
weightings  could  be  used.15 

•  Project  will  enable  significant  performance  improvements  in  an  MDA 
system — Weight  10. 

•  Likelihood  of  project  technical  success  is  high — Weight  5. 

•  Project  will  result  in  a  cost-effective  solution — Weight  5. 

•  Project  results  can  be  readily  adapted  into  future  systems  and  concepts  of 
operations — Weight  5. 

•  Project  will  face  minimal  nonmilitary  obstacles — Weight  5. 

A  suitable  scoring  range  would  then  be  defined  for  each  objective  and  used  for 
evaluating  the  project  alternatives.  Each  objective  would  typically  have  a  scoring  range 
of  1  to  10,  depending  on  specified  measures  or  standards.  At  this  stage  of  project 
definition,  the  scoring  ranges  primarily  reflect  qualitative  indicators.  For  example,  for 
Objective  1  (potential  performance  improvement),  the  scoring  range  might  be  from  1  for 
low  perfonnance  improvement  to  10  for  major  performance  improvement. 

The  next  step  is  to  score  each  alternative  for  the  objectives  using  the  agreed-upon 
scales.  These  individual  scores  are  then  multiplied  by  the  objective  weights  to  determine 
the  weighted  scores,  which  are  then  added  to  obtain  a  total  weighted  score  for  the  project. 
The  total  weighted  scores  are  then  used  as  a  primary  indicator  of  project  ranking. 

Table  V-4  gives  hypothetical  example  of  this  type  of  scoring  for  several  projects 
that  have  been  discussed  by  the  pilot  team. 


Table  V-4.  Example  Scoring  of  Three  S&T  Project  Alternatives 


Objective  (Score,  Weighted  Score) 

Performance  Cost-  Other 

Improvement  Low  Risk  effective  Adaptability  Issues 


(weight  10) 

(weight  5) 

(weight  5) 

(weight  5) 

(weight  5) 

Total 

New 

Sensors 

4 

40 

8 

40 

10 

50 

3 

15 

10 

50 

195 

New 

Propellant 

2 

20 

5 

25 

7 

35 

8 

40 

10 

50 

170 

New 

Booster 

8 

80 

2 

10 

10 

50 

10 

50 

6 

30 

220 

Weighted 


15  This  example  is  derived  for  the  case  of  evaluating  specific  technology  projects;  the  same  approach, 
with  some  modification  to  the  criteria  and  weights,  could  also  be  used  evaluate  and  rank  key 
technology  areas  or  alternative  missile  defense  system  concepts.  Scoring  could  be  done  by  a  team  of 
individuals,  and  the  results  averaged  to  produce  an  overall  score.  MUSTs  objectives  (e.g.,  the  project 
addresses  an  S&T  issue)  are  not  discussed  in  this  example. 


V-37 


In  this  example,  the  third  project,  low-cost  boost  technology  for  the  space-based 
laser,  received  the  highest  total  weighted  score,  and  thus  would  receive  the  highest 
ranking  recommendation  based  on  its  overall  total.  Typically,  the  highest  ranking 
projects  would  be  funded  in  descending  rank  order  until  the  available  funds  were 
exhausted.  However,  since  real-world  ranking  decisions  are  based  on  more  than  scores, 
the  scoring  model  only  provides  an  indication  of  rank  and  not  the  final  decision  by  the 
responsible  managers.  This  point  will  be  discussed  in  more  detail  later. 

Note  that  the  above  example  is  purely  hypothetical,  with  weights  and  scores 
chosen  arbitrarily.  Also  note  that  in  a  real-world  scoring  situation,  the  scoring  ranges 
would  be  better  defined,  and  the  rationale  for  the  scores  given  each  alternative  and 
objective  would  be  documented  to  better  understand  the  logic  behind  the  ranking 
recommendations. 

Although  the  scoring  model  in  Table  V-4  provides  a  means  of  ranking  the 
projects,  it  does  not  take  into  consideration  all  issues  of  importance.  In  particular,  the 
scoring  model  does  not  address  the  impact  the  projects  (or  technologies)  can  have  on 
other  MDA  systems. 

One  way  to  overcome  this  issue  is  to  weight  the  individual  projects  with  respect 
to  their  relevancy  to  the  MDA  systems,  whether  being  developed  or  only  conceptual. 
This  modification  is  illustrated  in  Table  V-5,  where  the  various  system  concepts  are 
weighted  from  0  to  10  points  based  on  their  importance  to  MDA.  The  relevance  of  each 
project  is  estimated,  and  these  weights  and  relevance  factors  are  then  multiplied  by  the 
raw  score  the  projects  received  from  the  previous  model  to  detennine  a  weighted 
relevance  for  each  system.  These  weighted  relevance  for  each  system  are  then  added  for 
each  project  to  yield  an  overall  relevance  score,  which  is  used  to  rank  the  projects. 

The  data  from  the  example  in  Table  V-5  indicate  the  highest  priority  project 
would  be  the  new  propellant  project,  based  on  its  widespread  relevance  to  a  number  of 
MDA  systems.  As  with  the  earlier  scoring  example,  this  case  is  purely  hypothetical,  and 
the  data  were  selected  to  demonstrate  the  effect  relevance  across  systems  has  on 
establishing  priorities.  Although  this  second  model  addresses  additional  factors,  the 
necessary  information  may  not  be  available  for  the  Gate  0  review.  Thus,  this  scoring 
model  method  may  be  more  applicable  for  Gate  1,  when  the  project  concept  has  been 
better  defined  and  can  be  more  readily  evaluated  in  terms  of  relevance  to  other  MDA 
systems. 
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As  the  project  moves  down  the  stage-gate  pipeline,  additional  information  and 
insight  will  be  developed  that  will  enable  the  use  of  more  quantitative  scoring  models. 
For  example,  in  later  stages  of  the  pipeline,  it  may  be  possible  to  quantitatively  assess  the 
potential  impact  of  the  S&T  project  versus  risks  (both  technical  and  implementation), 
thereby  enabling  more  realistic  trade-offs  when  establishing  priorities  between  project 
candidates.  However,  since  reliable  quantitative  infonnation  is  normally  not  available  in 
early  phases,  such  advanced  scoring  techniques  are  not  applicable. 

The  basic  Kepner-Tregoe  method  (and  variations)  mentioned  above  is  not  the 
only  scoring  technique  that  is  used  for  establishing  S&T  priorities.  Other,  more  elaborate 
weighting  schemes  have  also  been  created.  The  primary  motivation  for  developing  these 
advanced  approaches  is  to  provide  more  reliable  and  consistent  ways  to  analyze  complex 
issues  and  determine  relative  weights  of  decision-making  objectives. 

One  of  the  more  important  more  advanced  weighting  schemes  is  the  Analytical 
Hierarchy  Process  developed  by  Thomas  Saaty  in  the  1970s.  According  to  its  developer, 
the  Analytical  Hierarchy  Process  is 

A  method  for  breaking  down  a  complex,  unstructured  situation  into  its 
component  parts;  arranging  these  parts,  or  variables,  into  a  hierarchic 
order;  assigning  numerical  values  to  subjective  judgments  on  the  relative 
importance  of  each  variable;  and  synthesizing  the  judgments  to  determine 
which  variables  have  the  highest  priority  and  should  be  acted  upon  to 
influence  the  outcome  of  the  situation.16 


16 


Thomas  L.  Saaty,  Decision  Making  for  Leaders:  The  Analytic  Hierarchy  Process  for  Decisions  in  a 
Complex  World,  RWS  Publications,  Third  Edition,  2001. 
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Table  V-5.  Example  Scoring  of  Three  Hypothetical  S&T  Project  Alternatives  Against  IDA 
Pilot  Systems  (1  to  3)  and  Other  MDA  Applications  (4  to  12) 


New  Sensors 
(Raw  Score  195) 

New  Propellant 
(Raw  Score  170) 

New  Booster 
(Raw  Score  220) 

System 

Application 

Weight 
(0  to  10) 

Relevancy 
(0.0  to  1.0) 

Weighted 

Relevance 

Relevance 
(0.0  to  1.0) 

Weighted 

Relevancy 

Relevance 
(0.0  to  1.0) 

Weighted 

Relevance 

1.  BAAM  (Buried 
Autonomous 
Anti-Missile 
System) 

5 

1.0 

975 

1.0 

850 

0.0 

0 

2.  UAV 

(Unmanned 

Arial  Vehicle) 

5 

0.1 

98 

0.5 

425 

0.0 

0 

3.  SBL  (Space 
Based  Laser) 

5 

0.1 

98 

0.0 

0 

1.0 

1,100 

Subtotals 

1,171 

1,275 

1,100 

4.  ABL  (Airborne 
Laser) 

10 

0.1 

195 

0.5 

850 

0.0 

0 

5.  Ground  Based 
Midcourse 

10 

0.1 

195 

0.8 

1,360 

0.0 

0 

6.  Aegis  Ballistic 
Missile 

Defense 

10 

0.1 

195 

0.8 

1,360 

0.0 

0 

7.  THAAD 

(Theater  High 
Altitude  Air 
Defense 

System) 

10 

0.1 

195 

0.8 

1,360 

0.0 

0 

8.  Arrow 

5 

0.1 

98 

0.8 

680 

0.0 

0 

9.  Patriot  PAC-3 

10 

0.1 

195 

0.8 

1,360 

0.0 

0 

10.  MEADS 
(Medium 
Extended  Air 
Defense 

System) 

5 

0.1 

98 

0.8 

680 

0.0 

0 

11.  STSS  (Space 
Tracking  and 
Surveillance 
System) 

10 

0.1 

195 

0.0 

0 

1.0 

2,200 

12.  RAMOS 
(Russian- 
American 
Observation 
Satellite) 

5 

0.1 

98 

0.0 

0 

1.0 

1,100 

Subtotals 

1,464 

7,650 

3,300 

Totals 

2,635 

8,925 

4,400 
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The  Analytical  Hierarchy  Process  is  important  for  decision-making  in  complex 
situations  for  two  reasons.  First,  it  allows  the  user  to  structure  problems  in  an  integrated, 
hierarchical  manner  and  address  how  individual  elements  affect  the  performance  of  the 
entire  system.  Second,  it  provides  a  method  for  the  user  to  establish  priority  relationships 
between  competing  and  disparate  factors  in  a  logically  consistent  manner.  The  Analytical 
Hierarchy  Process  enables  this  function  by  using  matrixes  that  provide  pair-wise 
comparisons  of  the  properties  or  features  being  analyzed.  Combined,  these  two  features 
allow  the  method  to  be  used  to  define  and  evaluate  priorities  using  a  top-down, 
systematic  approach  and  a  bottom-up  view  that  provides  logically  consistent  criteria  and 
weightings.  These  dual  properties  are  particularly  useful  when  trade-offs  must  be  made 
between  conflicting  choices  that  are  difficult  to  compare  directly,  such  as  human  safety 
and  product  performance  or  environmental  consequences  and  economic  impact. 

The  Analytical  Hierarchy  Process  is  mentioned  here  because  it  is  increasingly 
being  touted  as  a  way  to  establish  priorities  in  difficult  planning  situations.  But  the 
method  it  is  a  complicated  decision-making  tool  that  can  be  very  expensive  to  design  and 
apply.  Consequently,  it  is  unlikely  that  the  Analytical  Hierarchy  Process  would  be  suited 
for  planning  early-stage  MDA  S&T  programs.  However,  under  certain  circumstances  the 
technique  may  be  applicable  to  MDA.  These  circumstances  could  include  analysis  and 
decision-making  for  complicated  architectural  and  layered  defense  alternatives, 
especially  during  later  phases  of  the  stage-gate  process. 

All  of  the  above  should  lead  the  reader  to  the  conclusion  that  the  scoring  methods 
used  to  assign  priorities  to  S&T  project  alternatives  need  to  be  selected  based  on  where 
the  projects  are  in  the  stage-gate  process.  Regardless  of  the  method  selected,  scoring 
models  are  not  a  panacea  for  good  decision-making.  Although  the  use  of  scoring  models 
helps  provide  a  rational  decision-making  process,  several  inherent  underlying  difficulties 
must  be  avoided.17  Occasionally,  scoring  models  are  used  that  do  not  yield  logical  or 
consistent  answers  or  fail  to  provide  enough  sensitivity  to  select  among  the  candidates. 
This  typically  means  that  the  problem  has  not  been  properly  framed  or  the  weightings 
have  not  been  adequately  derived,  thus  necessitating  a  redesign  of  the  scoring  model.  But 
no  scoring  method  will  be  perfect  because  no  model  can  accurately  mimic  reality,  capture 
all  relevant  infonnation,  or  consider  all  consequences.  In  the  end,  decision-making  is  still 


17 


James  G.  March,  A  Primer  on  Decision  Making:  How  Decisions  Happen ,  The  Free  Press,  1994. 
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a  human  process,  and  humans  are  notorious  for  behaving  differently  than  predicted  by 
decision  theory. 

To  guard  against  these  difficulties,  a  number  of  keys  to  successful  decision¬ 
making  have  been  developed.18  These  include  the  following: 

•  Use  a  value  creation  lens  for  developing  and  evaluating  opportunities. 

•  Clearly  establish  objectives  and  trade-offs. 

•  Discover  and  frame  the  real  problem. 

•  Understand  the  operational  impacts  of  the  decision. 

•  Develop  creative  and  unique  alternatives. 

•  Identify  experts  and  gather  meaningful  and  reliable  information. 

•  Embrace  uncertainty  as  the  catalyst  of  future  performance. 

•  Avoid  “analysis  paralysis”  situations. 

•  Use  systematic  thinking  to  connect  current  to  future  situations. 

•  Use  dialog  to  foster  learning  and  clarity  of  action. 

Because  the  use  of  scoring  models  will  become  an  integral  and  recurring  part  of 
the  MDA  stage-gate  process,  such  techniques  will  need  to  well  defined  and  documented. 
Their  use  will  also  require  training  and  discussions  between  the  project  champions  and 
the  decision-makers.  It  may  also  be  appropriate  to  assemble  ad  hoc  review  panels  of 
experts  to  score  the  project  proposals,  with  the  final  scoring  results  being  one  input  used 
by  the  gatekeepers  that  decide  whether  the  project  moves  to  the  next  stage-gate  phase. 

Implementation  of  scoring  models  into  the  MDA  stage-gate  process  will  require  a 
few  person-months  of  effort  to  define  the  models,  develop  appropriate  weights,  identify 
reviewers,  and  train  users.  It  is  assumed  that  this  function  would  be  the  responsibility  of 
the  individual  who  serves  as  the  process  owner  and  champion  for  MDA’s  stage-gate 
process.  Additional  resources  would  also  be  desirable  for  outside  support  for  model 
development  and  implementation  assistance.  The  level  of  resources  will  depend  on  the 
number  of  models  developed. 

Recommended  Next  Steps  for  MDA/AS 

The  above  material  and  the  data  contained  in  the  appendixes  represent  an  initial 
effort  to  compile  a  directory  of  analytical  tools  and  techniques  that  could  be  potentially 


18 


David  C.  Skinner,  Decision  Analysis:  A  Practitioner’s  Guide  to  Improving  Decision  Quality, 
Probabilistic  Publishing,  Second  Edition,  2001. 
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be  used  by  MDA/AS  in  planning  and  managing  its  S&T  program.  Collectively,  this 
information  should  be  viewed  as  a  first  step  in  defining  appropriate  tools.  Because  the 
topic  is  so  diverse  and  rapidly  changing,  the  collection  and  expansion  of  similar 
information  should  be  an  ongoing  effort  in  the  future. 

In  addition  to  this  ongoing  maintenance  of  a  tool  knowledge  base,  efforts  should 
also  be  undertaken  to: 

•  Designate  one  or  more  tool  focal  points — This  is  such  an  important  area  for 
MDA/AS  that  it  warrants  the  designation  of  one  or  more  individuals  to  serve 
as  focal  points  for  championing  the  development  and  use  of  the  types  of  tools 
discussed  in  this  report. 

•  Rank  the  tools  and  techniques — Each  tool  will  need  to  be  ranked  in  terms  of  its 
relevance  to  MDA/AS ’s  needs.  This  ranking  will  need  to  take  into 
consideration  related  tools  already  being  used  by  MDA  and  similar  techniques 
discussed  in  other  IDA  Phase  II  tasks. 

•  Develop  a  tool  implementation  plan — Based  on  the  above  rankings,  an 
implementation  plan  needs  to  be  developed  for  the  highest  priority  tools, 
including  estimates  of  resource  requirements.  (Specific  implementation  steps 
for  the  three  example  tools — stage-gate  processes,  scoring  models,  and 
modeling  and  simulation — are  discussed  in  previous  sections  of  this  report.) 

•  Implement  tools — Proceed  with  tool  implementation,  as  appropriate. 
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VI.  MODELING  AND  SIMULATION 


John  Meyer  and  Paul  Collopy 


INTRODUCTION 

Modeling  and  simulation  (M&S)  is  widely  recognized  as  a  tool  that  often  plays  a 
vital  role  in  R&D.  This  chapter  examines  how  M&S  can  be  used  within  MDA/AS  for 
S&T  program  planning,  execution,  and  management,  and  what  actions  are  needed  to 
move  forward  in  this  area.  It  includes  a  specific  example  of  a  combined  M&S  and 
scoring  method  being  developed  as  part  of  this  IDA  task  for  evaluation  of  missile  defense 
concepts  and  technologies:  The  Technology  Value  Model. 

M&S  is  the  development  and  use  of  various  models  and  simulators  to  investigate, 
understand,  or  provide  experimental  stimulus  for  both  existing  systems  that  cannot 
undergo  experimentation  due  to  resource,  security,  safety,  or  other  limitations  or  for 
conceptual  systems  and  technologies  that  do  not  exist.  A  “model”  is  a  mathematical, 
physical,  or  procedural  representation  of  a  system,  component,  technology,  entity, 
phenomenon,  or  process.  A  “simulation”  is  a  method  of  implementing  a  model  over  time. 
A  simulation  brings  the  model  to  life  and  shows  how  a  particular  object  or  phenomenon 
will  behave.  There  are  several  different  types  of  simulations,  including: 

•  Virtual  simulations  involving  totally  synthetic  (both  electronic  and  physical) 
representations  of  war-fighting  systems  and  environments  patterned  after 
actual  operations  and  equipment. 

•  Constructive  simulations  that  involve  real  people  providing  inputs  into  a 
simulation  that  carries  out  those  commands  by  exercising  artificial  models  of 
systems  and  people. 

•  Live  simulations  where  military  operations  are  carried  out  using  live  forces 
and  weapons  systems  on  test  or  exercise  ranges  to  simulate  actual  operating 
conditions. 

USES  AND  BENEFITS  OF  M&S  IN  S&T  PLANNING  AND  MANAGEMENT 

M&S  has  been  used  extensively  for  decades  in  many  areas  beyond  S&T  program 
planning,  execution,  and  management,  including  engineering  design,  manufacturing 
planning,  user  training,  business  exercises,  macroeconomic  and  financial  analysis,  and 


war  gaming.  Military  interest  in  M&S  involves  such  diverse  topics  as  requirements 
definition,  systems  interoperability,  test  and  evaluation,  battle-space  management, 
logistics,  and  training.  Military  M&S  is  typically  used  to: 

•  Reduce  time  to  field  new  or  upgraded  systems. 

•  Increase  military  worth  of  fielded  systems  while  simultaneously  optimizing 
force  structure,  doctrine,  tactics,  techniques,  and  procedures. 

•  Enable  concurrent  fielding  of  systems  with  their  training  devices. 

•  Reduce  total  ownership  costs  and  sustainment  burden  for  fielded  systems 
throughout  their  service  lives.1 

To  address  this  broader  military  context  of  M&S  during  the  past  few  years  DoD 
has  embraced  the  concept  of  simulation-based  acquisition.2  DoD  has  defined  it  as  “an 
acquisition  process  in  which  DoD  and  industry  are  enabled  by  robust,  collaborative  use 
of  simulation  technology  that  is  integrated  across  acquisition  phases  and  programs.”3  The 
concept  of  simulation-based  acquisition  has  an  even  broader  set  of  goals  than  those 
mentioned  earlier:4 

•  A  dramatically  improved  acquisition  process  enabled  by  the  application  of 
advanced  infonnation  technology. 

•  Earlier  and  better  informed  decisions  and  reduced  risk  by  more  accurate  and 
comprehensive  assessments  of  designs,  manufacturing,  support  and 
employment. 

•  The  early  optimization  of  system  performance  versus  total  cost  of  ownership. 

•  Lower  total  ownership  cost  and  standards-based  reuse  of  information  and 
M&S  software  to  minimize  their  costs. 

•  More  optimal  investments  enabled  by  system-of-systems  mission  area 
assessments. 

•  Enduring  collaborative  environments,  reusable  and  interoperable  tools,  and 
supporting  resources. 


1  Simulation  &  Modeling  for  Acquisition,  Requirements  and  Training  (SMART)  Reference  Guide,  Army 
Modeling  and  Simulation  Office,  April  2001. 

2  Roadmap  for  Simulation  Based  Acquisition,  Joint  Simulation  Based  Acquisition  Task  Force,  under  the 
Acquisition  Council,  DoD  Executive  Council  for  Modeling  and  Simulation  (EXCIMS),  December 
1998. 

3  Simulation-Based  Acquisition  Definition,  DoD  Modeling  and  Simulation  Acquitition  Council,  August 
2000. 

4  Ibid. 
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•  Automated  near-real-time  sharing  of  relevant  information  among  all  personnel 
with  a  need  to  know  by  use  of  distributed  product  descriptions  through  a 
common  technical  architecture  and  open,  commercial  data-interchange 
standards. 

The  tremendous  breadth  of  the  simulation-based  acquisition  concept  is  illustrated 
by  the  “to-be”  architecture  shown  in  Figure  VI- 1.  The  S&T  portion  of  the  process  model 
(in  the  lower  left  of  the  figure)  plays  a  limited  but  important  part  in  the  concept. 


Analyze 
Mission  & 
Alternatives 


Identify 
Mission  Needs 


Support 
Decision-Makers 
Warfighters 
Maintainers 


JWCA 

Make  Investment 
Decisions 


Develop 

Materiel 

Solutions 


\  *  - 
VIRTUAL  J  -jv 

PROTOTYPING  '  D9s|0n  s 

X 


Manufacture 

Product 


Manage 

Resources 


Define  ^ 
Functions 


Define  » 
Requirements 


Provide 
Capabilities y 


Figure  VI-1.  Simulation-Based  Acquisition  “To-Be”  Process  Model  and  Focus  of  S&T 

Activities 

(Source:  Adapted  from  “Roadmap  for  Simulation  Based  Acquisition,”  Joint  Simulation 
Based  Acquisition  Task  Force,  Acquisition  Council,  DoD  Executive  Council  for  Modeling 
and  Simulation  (DoD  EXCIMS),  December  1998.) 

In  addition  to  the  overall  role  S&T  plays  in  the  simulation-based  acquisition 
process  model,  the  reader  should  also  be  aware  of  the  various  types  of  models  that  are 
employed  in  M&S.  In  the  simulation-based  acquisition  context,  models  are  viewed  as 
part  of  a  hierarchy,  as  shown  in  Figure  VI-2.  Although  there  are  no  strict  differences 
between  types  of  models  shown  in  the  hierarchy,  they  do  tend  to  differ  considerably  in  a 
number  of  factors,  including  such  issues  as  objectives,  fidelity,  number  of  entities 
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incorporated  in  the  model,  and  cost,  as  shown  in  Table  V-6,  which  compares  some  of  the 
differences  between  S&T  models  and  product  engineering  models. 


Figure  VI-2.  Simulation-Based  Acquisition  Model  Hierarchy 

(Source:  Modified  from  “Roadmap  for  Simulation-Based  Acquisition,”  DoD  Executive 
Council  for  Modeling  and  Simulation  (EXCIMS)) 

It  is  also  important  to  note  that  the  simulation-based  acquisition  hierarchy  implies 
the  potential  for  compatibility  and  integration  up  and  down  the  hierarchy.  Thus,  systems 
models  could  conceivably  be  assembled  from  models  of  subsystems,  which  in  turn  could 
be  constructed  from  models  of  components,  which  could  be  based  on  physics-based 
models  of  the  underlying  technologies.  Although  such  a  hierarchical  “object-oriented” 
approach  to  M&S  is  far  from  being  a  reality,  it  is  one  of  the  major  goals  underlying  this 
acquisition  concept  and  offers  a  powerful  justification  for  moving  in  this  direction  in  the 
future. 


Table  VI-1.  Differences  between  S&T  Models  and  Engineering  Models 


S&T  Models 

Engineering  Models 

Objectives 

1 .  Analysis  of  alternatives 

2.  Sensitivity  analyses 

3.  Identification  of  key  variables 

4.  First-order  estimates  of 
performance  and  cost 

1.  Design  trade-offs 

2.  Refine  requirements  and  design 
details 

3.  Definition  of  subsystems 

4.  Performance  and  cost  verification 

Fidelity 

Low  to  moderate 

High 

Cost 

Low  to  high 

High 

Availability 

Very  limited  (custom) 

Limited  (may  use  common  approaches) 

Commercial 

Tools 

Few 

Many 

In  addition  to  variations  between  model  types,  it  is  also  important  to  note 


differences  between  who  develops  and  owns  the  models,  as  shown  in  Figure  VI-3.  Thus, 
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to  truly  integrate  the  models  within  the  simulation-based  acquisition  hierarchy  will 
require  considerable  collaboration  between  the  various  owners  and  stakeholders  involved 
in  the  process.  Because  most  M&S  efforts  have  been  undertaken  independently  in  the 
past  and  thus  represent  many  differing  approaches  and  instantiations,  this  integration 
within  the  hierarchy  and  across  a  system’s  life  cycle  will  represent  a  major  challenge  for 
many  years  to  come. 


In  terms  of  project  execution,  M&S  is  becoming  a  key  element  in  undertaking 
research  efforts.  Research  projects  generally  represent  an  iterative  optimization  process 
involving  (1)  hypothesis  or  concept  synthesis,  (2)  experimentation  and  evaluation,  and 
(3)  decisions  about  the  validity  of  the  hypothesis  or  concept.  However,  because  of  the 
expense  and  time  required  to  perform  the  experiments  and  evaluation  testing,  M&S  is 
being  increasingly  used,  as  a  separate  step  in  the  process,  to  investigate  the  hypothesis  or 
concept  prior  to  experimentation.  (This  was  certainly  true  of  the  concepts  explored 
during  the  Pilot  task  of  the  MDA  Phase  II  study,  during  which  relatively  simple  models 
were  extensively  used  to  evaluate  the  feasibility  of  the  concepts  and  identify  key  issues 
that  needed  to  be  addressed  in  follow-on  research.) 

In  addition  to  assessing  feasibility  and  performance  of  a  new  concept  or 
technology,  M&S  can  also  be  used  to: 

•  Determine  and  visualize  the  feasible  research,  design  space,  and  application 
envelope  for  a  new  concept  or  technology. 
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•  Identify  risks  and  significant  “showstoppers”  that  need  to  be  carefully 
examined  to  ensure  feasibility. 

•  Investigate  likely  costs  of  developing,  producing,  and  operating  the  solution. 

•  Address  issues  such  as  interoperability  and  user  interfaces. 

•  Explore  other  important  topics,  such  as  producibility,  reliability, 
maintainability,  logistics,  and  training,  if  appropriate. 

•  Provide  virtual  prototypes  and  technology  modules  that  can  be  used  in  larger 
simulations  and  integrated  computer-aided  exercises. 

•  Serve  as  an  important  mechanism  for  collaboration  between  technology 
developers  and  users. 

The  above  M&S  capabilities  provide  research  users  with  the  following  important 
benefits: 

•  Cost  savings  due  to  a  reduction  in  the  need  for  physical  prototypes  and  testing. 

•  Reduced  experimental  design  and  analysis  time. 

•  Increased  understanding  of  the  concept  or  technology  and  its  underlying 
phenomenology. 

•  The  ability  to  assess  more  options. 

•  Reduction  and  management  of  risks. 

In  addition  to  the  above  direct  benefits,  research  users  have  also  experienced  the 
following  indirect  benefits  from  M&S:5 

•  Generation  of  useful  surprises  or  knowledge  about  the  technology,  concept,  or 
applications. 

•  Greater  insight  into  organizational  needs  and  priorities. 

•  Better  understanding  of  the  consequences  of  S&T  management  decisions. 

•  Improved  innovations  due  to  the  more  extensive  interaction  among  appropriate 
stakeholders. 

M&S  Trends 

M&S  technology  and  applications  are  far  from  being  static.  Some  of  the  trends 
taking  place  include  the  following:6 


5 


M.  Shrage,  Serious  Play:  How  the  World’s  Best  Companies  Simulate  to  Innovate ,  Harvard  Business 
School  Press,  2000. 
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•  Development  of  models  incorporating  advanced  techniques  from  such  fields  as 
complexity  and  chaos  theory,  neural  networks,  agent-based  models,  response 
surface  methods,  and  genetic  algorithms  to  permit  analysis  of  complex, 
nonlinear  problems. 

•  Use  of  data  mining,  data  farming,  and  knowledge  discovery  tools  that  aid 
analysis  of  large  amounts  of  data  generated  from  simulations  and  use  of 
advanced  visualization  techniques  that  convey  useful  infonnation  from  data. 

•  Increasingly  pervasive  use  of  M&S  throughout  all  phases  of  a  weapon 
system’s  life  cycle  in  an  integrated  manner  to  achieve  simulation-based 
acquisition. 

•  Sharing  and  reuse  of  models  and  data,  among  domestic  users,  programs, 
industry,  and  Services  and  on  an  international  basis. 

•  Development  and  use  of  secure  interoperable  environments  and  standards, 
such  as  the  Defense  Modeling  and  Simulation  Office’s  High  Level 
Architecture  (now  IEEE  standard  1516),  to  help  bring  about  the  rapid 
federation  of  models  and  joint  simulations.7 

•  Increasing  cost  effectiveness  of  M&S  based  on  continued  advances  in 
computer  performance  and  reduced  cost  of  hardware  and  software. 

•  Emergence  of  relevant  commercial  tools,  such  as  MatLab,  Excel,  M&S 
languages,  and  some  of  the  integrated  modeling  environments  discussed 
earlier. 

•  Growing  experience  base  for  M&S  of  large,  complex  systems. 

Collectively,  these  trends  suggest  that  M&S  technology,  applications,  and 
infrastructure  are  moving  forward  rapidly  and  that  important  developments  can  be 
expected  in  the  next  few  years. 


6  L.  McGlynn  and  S.  Starr,  “SIMTECH  2007  ...  and  Beyond,”  Future  Modeling  and  Simulation 
Challenges,  NATO  Modeling  and  Simulation  Group  Conference,  Breda,  Netherlands, 
12-14  November,  2001. 

7  The  High  Level  Architecture  is  a  general-purpose  architecture  for  simulation  reuse  and 
interoperability.  It  was  developed  under  the  leadership  of  the  Defense  Modeling  and  Simulation  Office 
to  support  reuse  and  interoperability  across  the  large  numbers  of  different  types  of  simulations 
developed  and  maintained  by  the  DoD.  The  High  Level  Architecture  Baseline  Definition  was 
completed  on  August  21,  1996.  It  was  approved  by  the  Under  Secretary  of  Defense  for  Acquisition 
and  Technology  (USD(A&T))  as  the  standard  technical  architecture  for  all  DoD  simulations  on 
September  10,  1996.  Julien  Scharl  and  Dimitri  Mavris,  “Development  of  an  Object  Oriented  Vehicle 
Library  for  Automated  Design  Analysis,”  SAE  Paper  2001-01-3034,  2001. 
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Difficulties  in  Applying  M&S  in  S&T  Applications 

Although  M&S  offers  great  potential  in  S&T  program  planning,  execution  and 
management,  the  technology  also  raises  a  number  of  issues. 

For  example,  in  modeling  and  simulation  of  new  concepts,  a  trade-off  must  be 
made  between  model  accuracy  (i.e.,  fidelity)  and  efficiency  (i.e.,  development  costs  and 
computing  requirements).  For  technologies  and  concepts  that  are  early  in  their  S&T 
development,  detailed  information  and  models  are  unlikely  to  exist,  and  thus  new  models 
will  need  to  be  developed  or  existing  models  will  need  to  be  adapted  to  the  new  concepts. 
However,  development  of  high-fidelity  models  at  this  stage  of  concept  maturity  is  rarely 
possible  due  to  lack  of  data  or  understanding  of  the  underlying  phenomenology  and 
prohibitive  resource  requirements  (time  and  cost).  Development  of  detailed  models  based 
on  finite-element  analysis  and  computational  fluid  dynamics,  for  instance,  can  be  very 
expensive.  Similarly,  parametric  models  may  not  be  applicable  because  the  new 
technologies  and  concepts,  by  definition,  lie  outside  the  range  of  conventional  solutions. 

One  promising  solution  to  this  dilemma  is  the  development  and  use  of  “meta 
models.”8  Meta  models  are  “models  of  models”  that  represent  efficient  replacements  of 
the  basic  physics-based  models.  Meta  models  seek  to  fonn  a  more  compact 
representation  of  the  functional  relationships  between  model  inputs  and  outputs.  The 
meta  modeling  process  represents  a  functional  approximation  of  the  underlying  physics- 
based  model.  Many  techniques  exist  for  creating  such  meta  models,  from  linear 
regression  and  response  surface  methods  to  neural  networks. 

In  multidisciplinary  analysis  problems — which  represent  most  new  concepts  or 
technologies — separate  multiple  meta  models  corresponding  to  each  engineering  or 
technical  discipline  would  be  required.  The  term  “discipline”  refers  to  both  the  classical 
engineering  sense  (e.g.,  aerodynamics)  but  also  to  other  aspects  such  as  producibility. 
Most  likely,  the  underlying  physics-based  models  would  require  different  inputs  and  may 
even  operate  on  different  computing  platforms.  Thus,  some  type  of  integration 
environment  would  be  needed  to  facilitate  the  use  and  interaction  of  the  multiple  meta 
models.  Several  companies  have  developed  integration  environments  for  this  purpose, 


8  Ibid. 
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including  Engineous  Software’s  iSight  system9  and  Phoenix  Integration’s  Model 
Center.10 

Other  difficulties  in  using  M&S  techniques  for  S&T  applications  include  the 
following: 

•  Understanding  the  limitations  of  the  models — A  model  is  by  definition  not 
reality,  and  every  model  has  its  Achilles’  heel.  Understanding  the  assumptions 
used  in  constructing  the  model  and  the  limitations  and  risks  they  introduce  into 
the  results  of  the  simulation  is  an  important  factor  in  S&T  decision-making. 

•  Bugs,  flaws,  and  errors  in  the  model — Because  models  often  contain  bugs, 
flaws  and  errors,  it  is  important  that  they  undergo  some  type  of  verification, 
validation,  and  accreditation  (VVA)  before  their  results  are  taken  as  correct. 

•  Lack  of  standard  models — As  mentioned  earlier,  detailed  models  are  not  likely 
to  exist  before  they  are  needed  in  particular  S&T  applications.  Sometimes  this 
occurs  because  of  a  lack  of  agreement  among  users  about  what  such  models 
should  contain  and  how  they  should  be  constructed,  thereby  necessitating 
reinvention  of  the  model  each  time  it  is  needed.  In  other  cases,  there  is  a  lack 
of  knowledge  of  the  underlying  scientific  basis  on  which  to  create  an 
appropriate  model. 

Notwithstanding  these  difficulties,  M&S  will  surely  become  more  important  in 
S&T  applications  in  the  future. 

M&S  within  MDA/AS 

The  above  sections  have  addressed  the  use  of  M&S  in  research  in  general. 
However,  for  MDA/AS,  M&S  is  particularly  relevant  because  MDA’s  systems  are 

•  Large,  complex,  and  expensive  to  develop,  procure,  and  operate. 

•  Unique,  with  limited  legacy  data  and  complex,  often  qualitative  measures  of 
effectiveness. 

•  Difficult  and  expensive  to  physically  test. 

•  Complex  “systems  of  systems”  that  often  behave  in  difficult- to-predict  ways. 

•  Based  on  many  underlying  technologies  that  are  changing  at  a  rapid  rate,  in 
some  cases  more  rapidly  than  the  time  it  takes  to  develop  and  field  the  system. 
Moreover,  many  of  the  advanced  concepts  MDA  will  be  considering  in  the 


9 

y  www.engmeous.com. 

10  www.phoenix-int.com. 
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future  will  be  based  on  new  technologies  with  a  limited  understanding  of  the 
underlying  phenomenology. 

•  Being  applied  in  a  geopolitical  context  in  which  the  threats  they  are  designed 
to  counter  are  not  stable. 

For  all  of  the  above  reasons,  M&S  should  become  more  widely  used  within  MDA 
as  a  tool  for  evaluating  new  systems  concepts,  subsystems,  components,  and 
technologies.  The  question  then  becomes,  “How  does  MDA/AS  apply  this  technology  to 
its  needs?” 

To  answer  this  question,  the  IDA  Technology  Assessment  Pilot  experimented 
with  different  types  of  models  and  simulations  in  the  process  of  identifying  innovative 
new  concepts  for  missile  defense  that  could  serve  as  a  basis  for  defining  a  long-tenn  S&T 
program.11  The  definition  and  analysis  of  each  concept  involved  some  type  of  modeling 
and  simulation  to  establish  the  preliminary  feasibility  of  the  concept  and  to  identify  the 
underlying  design  issues  and  technologies  that  will  have  the  greatest  effect  on  the 
concept’s  perfonnance  and  viability. 

These  models  and  simulations  were  custom  developed  as  low-cost  analysis  tools 
because  suitable  off-the-shelf  tools  were  not  available.  An  MDA/AS  toolkit  consisting  of 
suite  of  basic  models  and  simulation  environments  would  have  been  useful,  and  such  an 
M&S  toolkit  would  undoubtedly  prove  to  be  a  valuable  asset  to  MDA  and  its  contractors 
in  the  future.  A  brief  literature  search  showed  that  some  models  already  exist  for  space 
and  missile  applications.12  Some  of  the  potentially  relevant  models  are  summarized  in 
Appendix  D.  Undoubtedly,  other  relevant  models  are  available  from  proprietary  and 
commercial  sources.  In  addition,  numerous  existing  models  address  related  areas,  such  as 
command-and-control  systems,  atmospheric  conditions,  and  integrated  battle 
management. 

The  IDA  Technology  Assessment  Pilot  also  experimented  with  developing  a 
more  robust  M&S  tool  for  assessment  of  technologies  and  concepts.  This  tool — The 
Missile  Defense  Technology  Value  Model — is  summarized  in  the  next  section. 


11  Van  Atta  et  al,  “Results  of  a  Technology  Assessment  Pilot  Project  for  MDA.” 

12  Since  no  analysis  has  been  undertaken  about  the  relevancy  of  these  models,  it  is  unclear  to  what  extent 
they  could  be  useful  to  MDA/AS  in  the  planning  and  management  of  an  S&T  program.  They  are 
presented  here  only  to  illustrate  the  breadth  of  existing  capabilities. 
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M&S  EXAMPLE:  THE  MISSILE  DEFENSE  TECHNOLOGY  VALUE  MODEL 


The  Missile  Defense  Technology  Value  Model  uses  attributes  of  technologies  to 
assess  their  value.  Technologies  are  evaluated  by  comparing  the  value  of  a  ballistic 
missile  defense  system  (BMDS)  when  it  incorporates  the  technology  with  the  value  of  a 
baseline  BMDS  without  the  technology.  The  difference  is  the  gross  value  of  the  fielded 
technology.  Adjustments  are  made  for  the  time  and  cost  required  to  develop  the 
technology  and  the  risk  that  the  technology  will  not  work.  The  result  is  the  net  value  of 
the  technology.  Technologies  can  be  ranked  as  investment  opportunities  based  on  their 
net  value. 

The  core  of  the  model  is  a  mathematical  representation  of  a  BMDS.  This  tool  is 
important  beyond  the  immediate  purpose  of  technology  evaluation.  The  value  model  can 
assess  whether  one  candidate  missile  defense  system  design  is  better  than  another.  That 
is,  it  formalizes  the  notion  of  better  with  regard  to  missile  defense  system  design.  Any 
attempt  to  design  the  best  missile  defense  require  a  sound  notion  of  what  it  means  to  say 
that  one  design  is  better  than  another.  Thus,  the  value  model  (or  a  functional  equivalent) 
is  a  prerequisite  to  creating  the  best  possible  missile  defense  system. 

For  example,  the  value  model  can  serve  as  the  objective  function  for  optimizing 
the  conceptual  design  of  a  missile  defense  system.  That  is,  optimization  becomes  the 
search  for  the  highest  value  system,  as  assessed  by  the  value  model.  Trade  studies  can 
use  the  value  model  to  assess  alternatives.  The  trade  option  that  leads  to  the  highest 
valued  system,  per  the  value  model,  is  the  preferred  alternative.  More  important,  in  the 
detailed  design  phase,  optimization  objectives  for  each  component  can  be  derived  in  a 
straightforward  manner  using  the  value  model.  The  result  is  distributed  optimal  design, 
which  is  distinctly  different  from  the  state-of-the-art  specification  flow-down  process  that 
repeatedly  leads  to  systems  that  are  overweight  and  over  cost  during  detailed  design.13 

All  these  applications  (concept  optimization,  trade  study  evaluation,  and 
distributed  optimal  design)  are  possible  with  the  Missile  Defense  Technology  Value 
Model.  However,  the  specific  goal  of  this  particular  model  is  technology  evaluation.  The 
Missile  Defense  Technology  Value  Model  describes  technology  in  terms  of  performance 
and  cost  metrics  to  deduce  measures  of  benefit  and  cost  that  are  then  combined  to  yield 


13  Paul  D.  Collopy,  “Economic-Based  Distributed  Optimal  Design.”  AIAA  Paper  2001-4675,  American 
Institute  of  Aeronautics  and  Astronautics,  Reston,  Va .,  2001. 
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net  economic  value.  It  scores  technologies  in  a  way  that  is  objective,  repeatable, 
consistent,  and  transparent: 

•  Objective — Value  model  scores  are  based  on  perfonnance  and  cost  metrics 
that,  if  the  technology  were  in  production,  could  be  physically  verified. 

•  Repeatable — The  value  model  is  essentially  a  deterministic  mathematical 
function.  Unlike  Dephi  techniques,  quality  function  deployment,  or  other 
methods  that  depend  on  a  group  of  people  arriving  at  a  consensus,  the  value 
model  will  always  give  precisely  the  same  score  to  a  technology  with  the  same 
metrics. 

•  Consistent — If  any  good  metric  is  increased,  the  score  from  the  model  will 
never  decrease.  Thus,  when  comparing  two  technologies,  if  one  is  the  same  or 
better  on  every  metric,  it  will  have  the  same  or  better  score. 

•  Transparent — When  one  technology  outscores  another,  the  reasons  for  the 
scoring  difference  can  be  easily  understood  in  tenns  of  differences  in  the 
technologies’  metrics. 

The  value  model  characterizes  a  technology  by  a  set  of  metrics.  Technologies  that 
use  the  same  metrics  are  grouped  together  into  technology  buckets.  For  example,  booster 
rocket  structural  technologies  are  characterized  by  their  impact  on  rocket  mass  ratio  and 
cost.  All  technologies  include  technology  readiness  level,  probability  of  successful 
development,  and  cost  of  development  as  metrics.  A  technology’s  metrics  are  directly 
used  to  in  the  value  model  to  assess  the  value  of  the  technology.  Below  is  an  example  of 
a  technology  template  to  input  the  metrics  for  technologies  in  a  single  bucket,  kill  vehicle 
flight  control. 
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Kill  Vehicle  Flight  Control 


Name  Value 


Description 


Designation 

Designation 

Performance 

Dead  Miss 
Dead  Accel 
Live  Miss 
Maneuver 
Sensor  Lag 

Readiness 

TRL 

Cost 

Development 


Mean  miss  distance  for  ballistic  target 
Maximum  acceleration  reqd  for  Dead  Miss 
Mean  miss  distance  for  maneuvering  target 
Maneuver  ratio  required  for  live  miss 
Sensor  lag  assumed  in  performance  metrics 


Technology  Readiness  Level 


Cost  of  SDD  through  IOC 


Units 


m 

m  /  sec2 
m 

none 

sec 


see  table 


$  000  000 


Figure  VI-4.  Example  of  Technology  Template 


Kill  chains  separate  the  process  of  intercepting  the  target  into  a  series  of  steps,  all 
of  which  are  necessary  for  successful  interception.  A  probability  of  success  is  associated 
with  each  step,  and  the  probability  of  intercepting  the  target  (probability  of  kill)  is  the 
product  of  all  the  step  probabilities.  Although  a  more  complex  relationship  between  step 
probabilities  and  probability  of  kill  can  be  accommodated  in  the  model,  it  has  not  been 
necessary  in  the  prototype. 


Most  of  the  cost  of  the  interceptor  can  be  associated  with  steps  in  the  kill  chain. 
Therefore,  each  kill  chain  step  calculates  a  step  probability  and  a  cost.  The  system  kill 
probability  and  system  cost  are,  roughly,  the  product  of  the  step  probabilities  and  the  sum 
of  the  step  costs. 


Figure  VI-5  shows  the  kill  chain  for  the  BAAM  system  used  for  the  prototype 
value  model  and  kill  chains  for  other  BMDS  layers.  (More  detail  on  the  BAAM  concept 
appears  in  the  companion  volume  on  the  Technology  Assessment  Pilot  Project.)  The 
black  dots  are  the  kill  chain  steps  for  the  concept.  Note  that  because  many  steps  are 
common  to  many  different  systems,  the  prototype  model  has  already  developed  many  of 
the  necessary  modules  for  value  models  for  other  missile  defense  concepts. 
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BAAM  deployed  before  launch 

Boost  phase 
intercept 

Airborne 

laser 

Pulsed 

cxcimcr  laser 

BAAM 

• 

Midcoursc 

intercept 

Terminal 

intercept 

Rapidly 

deployable 

interceptor 

Ground  /  sea  interceptor  deployed  in  kill  zone 

• 

• 

• 

Aircraft  /  satellite  stationed  in  kill  zone 

• 

• 

BAAM  deployed  in  kill  zone 

• 

Deployment  rockets  based  in  kill  zone 

Launch  detected 

• 

• 

• 

• 

• 

• 

Low  altitude  tracking  (below  horizon) 

• 

• 

• 

Off-board  high  altitude  tracking 

• 

• 

• 

Decision  to  intercept 

• 

• 

• 

• 

• 

Target  discrimination 

• 

Deployment  to  interceptor  kill  zone 

Interceptor  boosted  to  target 

• 

• 

• 

• 

On  board  tracking 

• 

• 

• 

• 

Tracking  laser  illuminating  target 

Beacon  laser  illuminating  target 

Weapon  laser  on  target 

Weapon  laser  dwell  time  sufficient 

Laser  damage  defeats  attack 

Kill  vehicle  maneuvers  to  intercept 

• 

• 

• 

• 

• 

Interceptor  defeats  attack 

• 

• 

• 

• 

• 

Figure  VI-5.  Kill  Chains  for  Different  BMDS  Systems  Concepts 

The  Missile  Defense  Technology  Value  Model  is  implemented  as  a  Microsoft 
Excel  spreadsheet.  The  bulk  of  the  model  is  devoted  to  calculating  the  net  value  of  a 
BAAM  system  in  service.  When  a  technology  is  input  into  the  system,  quantitative 
metrics  of  the  technology  change  attributes  of  the  system  and  therefore  change  its  net 
value.  The  change  in  system  net  value  is  the  gross  value  of  the  technology  to  the  system. 

As  explained  below,  technologies  are  evaluated  under  10  different  scenarios,  each 
representing  different  hostile  attacks.  Scenarios  are  characterized  by  the  value  of  the  asset 
threatened  by  the  missile  in  terms  of  the  lives  at  risk,  the  missile  trajectory,  and  physical 
aspects  of  the  missile  as  a  target.  Scenarios  are  implemented  as  separate  columns  in  the 
model  spreadsheet.  Each  scenario  calculates  probability  of  kill  (the  probability  of  a  single 
interceptor  defeating  one  target  missile)  and  interceptor  unit  cost. 

Probability  of  kill  and  unit  cost  are  combined  into  a  cost-per-kill  metric.  Together 
with  the  asset  value,  cost  per  kill  determines  the  net  value  of  the  system.  Each  scenario  is 
then  weighted  by  the  probability  of  it  actually  occurring.  The  weighted  net  values  are 
summed  to  give  the  overall  net  value  of  the  system.  The  difference  between  this  overall 
value  and  a  reference  value  is  the  gross  value  of  the  technology.  This  is  adjusted  for 
technology  development  to  yield  the  net  value  of  the  technology,  which  is  the  ultimate 
technology  assessment. 
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The  first  step  in  the  model  is  detennining  the  value  of  the  threatened  asset.  The 
asset  is  characterized  by  the  number  of  deaths  anticipated  in  the  scenario.  The  deaths  are 
translated  into  dollars.  Military  and  civilian  deaths  are  considered  equivalent  in  this 
conversion.  Next,  the  minimum  number  of  BAAM  emplacements  is  determined,  based  on 
the  spread  of  threat  trajectory  azimuths  from  the  specific  threat  location.  For  example, 
North  Korean  missile  threats  to  the  continental  United  States  spread  over  an  azimuth  of 
about  70°.  The  actual  number  of  interceptors  deployed  would  also  depend  on  the  number 
of  threat  missiles  to  be  simultaneously  launched  and  the  cost  per  kill  of  the  interceptor. 
Next,  the  launch-detection  process  is  modeled,  yielding  the  probability  that  the  launch  is 
detected  and  the  time  required  for  detection.  This  time  is  added  to  the  time  required  for 
fire  control  (there  is  no  external  decision  loop  in  the  BAAM  system)  to  yield  the  time 
delay  between  target  launch  and  interceptor  launch. 

The  next  series  of  steps  characterize  the  interceptor.  First,  the  interceptor 
trajectory  is  determined  by  simultaneously  solving  for  the  point  of  intercept  and  the 
standoff  distance  between  the  target  launch  and  the  interceptor  emplacement.  The  delay 
between  target  and  interceptor  launch  is  the  input  to  the  iteration.  Once  the  trajectory  is 
determined,  the  kill  vehicle  velocity  at  intercept  is  determined.  The  kill  mass  is  then  sized 
based  on  the  kinetic  energy  required  for  kill.  This  sizes  the  kill  vehicle,  which,  taken  with 
the  trajectory,  sizes  the  interceptor  booster.  Cost  of  the  interceptor  is  inferred  from  the 
size  of  the  components. 

The  next  step  determines  the  probability  of  the  kill  vehicle  defeating  the  target 
missile.  The  last  step  in  examining  the  system  operation  assesses  the  cost  and  probability 
of  success  of  deploying  the  BAAM  system  prior  to  launch.  This  is  done  last  because  the 
size  of  the  missile  and  standoff  distance  from  the  target  launch  site  are  already 
determined. 

Probabilities  of  success  from  each  step  are  combined  into  an  overall  probability  of 
kill,  and  cost  of  components  is  rolled  up  into  an  interceptor  unit  cost.  These  parameters 
for  a  one-on-one  engagement  are  combined  into  a  cost-per-kill  metric  that  is  directly 
used,  with  the  asset  value,  to  calculate  the  net  value  of  a  many-on-one  (or,  in  some 
scenarios,  many-on-many)  engagement.  From  the  net  value  of  the  system  in  each 
scenario,  the  overall  net  value  of  the  system  is  detennined,  and  the  gross  value  of  the 
technology  is  calculated.  The  model  corrects  this  gross  value  to  a  net  present  value  of  the 
technology  investment  opportunity.  Considerations  are  the  technology  readiness  level,  its 
probability  of  success,  and  the  cost  required  to  develop  the  technology. 
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In  three  areas,  considerable  work  was  done  to  establish  a  rigorous  foundation  for 
the  equations  and  coefficients  used  in  the  value  model: 

•  Cost  per  kill — A  theory  has  been  developed  for  combining  the  probability  of 
an  interceptor  defeating  a  target  missile  with  the  cost  of  the  interceptor  to  yield 
a  summary  cost  per  kill  metric.  Together  with  the  value  of  the  asset  under 
attack,  cost  per  kill  directly  yields  the  net  value  of  the  ballistic  missile  defense 
system. 

•  Scenarios — To  establish  the  value  of  the  assets  that  might  be  attacked  by 
hostile  missiles,  a  set  of  representative  scenarios  has  been  developed  using 
different  types  of  conflicts,  different  classes  of  missiles,  and  different  types  of 
warheads. 

•  Threat  missiles — Threat  missiles  have  been  characterized  for  each  of  the  attack 
scenarios. 

We  wish  to  evaluate  an  interceptor  characterized  by  unit  cost,  s,  and  the 
probability  that  it  will  kill  a  target  missile,  pk .  Interceptors  can  be  launched  in  squadrons. 
A  squadron  attack  of  n  interceptors  is  described  by  the  probability  tree  in  Figure  VI-6: 


Target  killed 


Target  survived 


Figure  VI-6.  Probability  Tree  for  Squadron  Attack 

Note  that  the  interceptors  are  launched  essentially  simultaneously,  not  in  a  shoot- 
look-shoot  manner.  This  can  be  appropriate  within  a  layer  of  missile  defense,  particularly 
boost-phase  defense,  but  may  not  apply  to  a  multilayer  system  of  systems. 

Attacks  are  assumed  to  be  probabilistically  independent.  Each  node  represents  the 
attack  of  one  interceptor  on  the  target  missile.  Every  upper  branch  describes  an  instance 
where  the  target  is  killed.  The  target  only  survives  the  squadron  attack  if  it  survives  the 
attack  by  every  interceptor.  This  outcome  is  reached  only  by  following  the  lower  branch 
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leaving  each  node.  The  probability  that  the  target  survives  is  the  product  of  the 
independent  probabilities  on  each  of  the  lower  nodes. 

If  the  value  of  the  assets  that  would  be  destroyed  by  the  enemy  missile  is  V,  and 
the  interceptor  unit  cost  is  s,  the  net  value  of  the  attack  is  the  expectation  of  the  value  of 
attacking  the  target  missile  minus  the  cost  of  all  the  interceptors.  The  most  effective 
squadron  is  one  where  n,  the  number  of  interceptors,  is  chosen  to  maximize  the  net  value 
of  the  attack.  This  would  find  the  best  balance  between  the  cost  of  the  interceptors  and 
the  probability  of  destroying  the  target. 

It  is  interesting  to  note  that  the  expected  value  of  the  optimal  attack  depends  only 
on  the  asset  value  and  the  cost  per  kill.  It  turns  out  that  the  system  with  the  lowest  cost 
per  kill,  when  used  optimally,  yields  the  greatest  net  value,  no  matter  what  the  value  of 
the  protected  asset  is.  Therefore,  if  a  missile-defense  system  is  designed  to  minimize  cost 
per  kill,  it  will  provide  optimal  value.  This  is  not  true  of  traditional  formulations  of  cost 
per  kill. 

Our  value-modeling  effort  employs  a  limited  set  of  threat  scenarios  (15)  as  a  basis 
for  evaluating  the  potential  contribution  of  technology  developments  coming  out  of  the 
Pilot  Technology  Assessment  effort  (Table  VI-2).  These  scenarios  are  designed  with  a 
view  toward  spanning  the  space  of  relevant  missile  defense  challenges,  while  also 
focusing  attention  on  the  set  of  challenges  deemed  most  likely.  They  can  also  serve  as  a 
basis  for  formulating  a  time-phased  implementation  plan  aimed  at  providing  growing 
capability  against  the  most  daunting  challenges  in  the  mission  space. 
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Table  VI-2.  Technology  Value  Model  Scenarios 


Scenarios  1-4 

New  York  City  attacked  with  [1  or  3]  single-warhead  SCUD  B  missiles  fired  from 
320  km  away  in  the  Atlantic  Ocean,  carrying  a  payload  of  either  a  [10  kt  nuclear 
warhead  or  700  kg  Sarinjwith  a  circular  error  probable  (CEP)  of  3  km  and  no  spin 
stabilization  of  the  warheads. 

Death  toll:  Nuclear:  110,000  Chemical:  30,000 

Scenario  5 

New  York  City  attacked  with  two  MIRVed  (multiple  independently  targeted  reentry 
vehicle)  SS-18,  carrying  ten  500  kt  warheads,  fired  from  7,000  km  away  in  Russia, 
with  10  decoys  with  similar  ballistic  coefficients  as  the  warheads,  50  mylar  balloons, 
and  20  kg  of  chaff. 

Death  toll:  8,000,000 

Scenarios  6-7 

The  major  cities  on  the  U.S.  West  Coast — San  Diego,  Los  Angeles,  San  Francisco, 
and  Seattle — as  well  as  the  U.S.  interceptor  base  in  Alaska,  each  attacked  near- 
simultaneously  with  [2  or  10]  missiles  each,  fired  from  10,000  km  away  in  China, 
with  each  missile  carrying  four  200  kt  nuclear  warheads  and  employing  10  decoys 
with  similar  ballistic  coefficients  as  the  warheads,  50  mylar  balloons,  20  kg  of  chaff 
with  CEP  of  2  km. 

Death  toll:  2  missiles  per  city:  1,800,000;  10  missiles  per  city:  9,000,000 

Scenarios  8-9 

From  3,500  km  away,  North  Korea  launches  a  single-warhead  missile  at  Andersen 
Air  Force  Base  on  Guam.  The  missile  carries  a  20  kt  nuclear  warhead  and  [no 
countermeasures  or  employing  5  decoys  with  similar  ballistic  coefficients  as  the 
warheads]  with  CEP  of  4  km. 

Death  toll:  10,000 

Scenarios  10-13 

Tokyo  attacked  with  [2  or  20]  single-warhead  missiles  fired  from  1,200  km  away  in 
North  Korea.  Each  missile  carries  a  [(20  kt  nuclear  warhead  or  700  kg  Sarin],  with  no 
countermeasures. 

Death  toll:  2  nuclear:  100,000;  20  nuclear:  1 ,000,000 

Chemical:  2  chemical:  26,000;  20  chemical:  260,000 

Scenarios  14-15 

[Central  Command  (CENTCOM)  headquarters  in  Qatar  or  Tel  Aviv]  attacked  with 
two  single-warhead  SCUD  B  missiles  fired  from  300  km  away  (CENTCOM)  or  from 
1,300  km  away  (Tel  Aviv)  in  Iran.  Each  missile  carries  a  payload  of  a  10  kt  nuclear 
warhead  (CENTCOM)  or  200  kg  Anthrax  (Tel  Aviv)  with  no  countermeasures  and  a 
CEP  of  1  km. 

Death  toll:  CENTCOM:  8,000  Tel  Aviv:  135,000 

Analyses  of  the  missile-defense  problem  focus  on  three  critical  challenge 
dimensions: 


•  Threat  size — The  number  of  missiles/warheads. 

•  Threat  location — Uncertainty  as  to  the  location/range  from  which  the  missiles 
may  be  fired. 

•  Threat  sophistication — The  complexity  of  countermeasures  employed. 
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A  recent  RAND  report14  represented  these  dimensions  in  Figure  VI-7  below. 


Easiest 


Threat  location 
uncertainty 


Figure  VI-7.  Dimensions  of  the  ballistic  missile  threat  space 

In  addition,  for  the  purposes  of  value  modeling,  we  need  to  specify  the  potential 
targets  of  an  attack  and  the  type  of  warhead  being  carried,  because  these  will  affect  the 
consequence  (negative  value)  of  failing  to  intercept  a  particular  missile/warhead  aimed  at 
a  particular  target. 

In  2000,  the  CIA  released  an  unclassified  version  of  its  National  Intelligence 
Estimate  titled  Foreign  Missile  Developments  and  the  Ballistic  Missile  Threat  through 
2015. 15  That  estimate  cited  the  following  central  threats  to  the  United  States: 

•  Medium  or  short-range  missiles  launched  at  U.S.  cities  from  the  sea. 

•  Long-range  missiles  with  nuclear  or  chemical  warheads,  launched  from  North 
Korea  or  China  at  regional  allies  or  the  west  coast  of  the  United  States. 

•  Medium  and  short-range  missiles  launched  from  Iran  or  Syria,  with  chemical 
warheads  or  possibly  nuclear  warheads,  in  the  future,  targeting  U.S.  interests 
and  forces  in  the  region. 

The  estimate  did  not  indicate  the  relative  likelihood  of  these  threats  emerging  or 
being  realized.  As  for  terrorist  threats,  the  estimate  considered  that  such  attacks  would 
most  likely  be  delivered  by  means  other  than  missiles.  With  these  assumptions  as  general 
guidance,  the  specific  scenarios  in  the  table  below  were  used  for  purposes  of  analysis. 


14  David  C.  Gompert  and  Jeffrey  A.  Isaacson,  “Planning  a  Ballistic  Missile  Defense  System  of  Systems: 
An  Adaptive  Strategy,”  RAND  Issue  paper  IP-181,  1999. 

http://www.cia.gov/nic/pubs/other_productsAJnclassifiedballisticmissilefmal.pdf. 
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The  American  Physical  Society  report  on  boost-phase  intercept  is  used  to 
realistically  characterize  the  threat  in  an  unclassified  environment.16  Different  target 
missile  designs  are  assumed  for  each  scenario,  with  the  missile  sized  to  the  range 
required  in  the  scenario.  Missiles  are  assumed  to  follow  optimal  Keplerian  ballistic 
trajectories.  In  boost  phase,  the  missiles  execute  gravity  turns  from  a  vertical  launch. 
Target  missiles  are  assumed  to  have  a  specific  impulse  of  270  seconds  and  a  booster  mass 
ratio  of  90%,  consistent  with  a  medium  technology,  low-cost  kerosene  and  liquid  oxygen 
propellant.  Acceleration  at  launch  is  1.5  g  for  short-range  missiles  (less  than  1,000 
kilometers)  and  1 .0  g  for  longer  range  missiles.  Thrust  is  constant  throughout  each  stage 
bum.  All  targets  are  two-stage  missiles  with  a  1  metric  ton  warhead. 

The  essence  of  economic  evaluation  of  a  missile  defense  system  is  the  trade 
between  the  cost  of  the  ballistic  missile  defense  system  and  improvement  in  the 
probability  of  defeating  enemy  missiles.  Given  scenarios  that  project  casualty  rates  for 
enemy  missile  attacks,  the  trade  boils  down  to  dollars  versus  lives.  How  much  should  be 
spent  to  protect  a  large  number  of  people  from  death  at  the  hands  of  a  political  enemy? 
Both  economic  assessments  of  the  impact  of  terrorist  attacks  and  society’s  willingness  to 
invest  in  a  response  to  such  attacks  reveal  that  the  relationship  between  dollar  value  and 
lives  lost  is  not  linear.  For  this  model,  the  value  of  life  is  captured  by  the  exponential 
curve  in  Figure  VI-8.  For  a  very  small  number  of  lives,  the  value  is  $50  million  per  life. 
For  very  large  casualties,  the  value  approaches  $3  million  per  life.  This  figure  is  based  on 
the  norm  used  in  U.S.  courts  for  civil  awards  in  liability  cases,  which  is  just  under  $2 
million,  plus  a  50%  addition  to  account  for  destruction  of  infrastructure  in  addition  to  the 
loss  of  life.  The  other  significant  assumption  relates  to  the  value  of  time:  a  real  discount 
rate  of  5%  is  used  to  account  for  the  disadvantage  of  long  periods  of  technology 
development. 


16  Lamb  and  Kleppner,  2003. 
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Figure  VI-8.  Kill  Chains  for  Different  BMDS  Systems  Concepts 


Quantitative  Analysis  with  the  BAAM  Technology  Value  Model 

Although  the  current  study  did  not  include  a  technology  search  to  obtain  metrics 
of  specific  technologies  which  could  be  evaluated,  the  Technology  Value  Model 
nevertheless  provides  some  broad  direction  for  promising  areas  of  S&T  investment. 

For  example,  some  performance  attributes  are  clearly  of  minor  importance.  A  10- 
second  improvement  in  specific  impulse  of  the  interceptor  rocket  would  be  a  very 
challenging  goal  based  on  progress  made  over  the  last  two  decades.  Furthermore,  it 
would  only  provide  a  $5  million  increase  in  value  to  the  BAAM  weapon  system. 
Likewise,  a  1%  decrease  in  the  drag  coefficient  of  the  rocket’s  aerodynamic  shell 
provides  only  $0.1  million  of  value. 

Rocket  mass  ratio  (ratio  of  fuel  mass  to  total  rocket  mass)  is  a  more  promising 
research  area.  Increasing  the  mass  ratio  from  85%  to  86%  is  worth  $1  billion.  The 
baseline  interceptor  rocket  weighs  about  100  kilograms.  A  1%  improvement  in  mass  ratio 
corresponds  to  reducing  the  weight  of  the  structure  in  the  missile  by  1  kilogram. 

A  common  design  issue  is  the  trade  between  weight  and  manufacturing  cost.  This 
trade  determines  whether  exotic  materials  should  be  used  to  reduce  weight.  The  value 
model  suggests  that  for  the  100  kilogram  baseline  interceptor,  a  $2  million  increase  in 
manufacturing  cost  corresponds  to  a  $1  billion  loss  in  system  value.  Therefore, 
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expending  up  to  $2  million  in  manufacturing  to  reduce  the  structural  weight  of  the 
interceptor  by  1  kilogram  would  be  a  positive  trade.  BAAM  is  very  much  in  the  realm  of 
exotic  structural  material  applications.  Although  the  value  of  technology  improvements 
(the  $  1  billion  figure)  is  sensitive  to  scenario  assumptions,  the  trade  factor  ( 1  kilogram  ~ 
$  2  million)  is  fairly  independent  of  them. 

Often  the  value  of  a  technology  is  very  sensitive  to  the  current  state  of  the 
technology.  Figure  VI-9  below  shows  the  value  of  the  BAAM  system  versus  the 
probability  that  the  BAAM  unit,  dropped  from  the  delivery  aircraft,  will  emplace  in  the 
ground  in  such  a  way  that  the  interceptor  can  be  effectively  launched.  If  the  probability  of 
success  is  zero,  the  overall  system  has  no  value.  As  the  probability  exceeds  20%,  the 
sensitivity  of  system  value  to  probability  of  emplacement  (the  slope  of  the  curve)  is 
greatly  reduced.  However,  the  overall  value  of  BAAM  is  very  large:  $1.8  trillion  in  the 
baseline  case.  Even  when  the  probability  of  emplacement  is  35%,  the  value  of  an  increase 
of  1%  is  over  $1  billion.  As  the  probability  reaches  90%,  the  value  of  an  additional  1% 
improvement  is  reduced  to  $10  million,  which  would  make  it  a  marginal  S&T  prospect. 

2,000 


BAAM 
System  Value 
($  billions) 


0%  20%  40%  60%  80%  100% 

Probability  of  Successful  Emplacement 

Figure  VI-9.  Value  of  the  BAAM  System  Versus  Emplacement  Probability 

The  sensor  used  by  BAAM  to  detect  target  missile  launch  is  assumed  to  have  a 
baseline  sensitivity  of  4  standard  deviations — that  is,  the  mean  of  the  signal  exceeds  the 
mean  of  the  background  noise  by  4  standard  deviations  of  the  background  noise 
distribution.  Increasing  the  sensitivity  by  1  more  standard  deviation  is  worth  $600 
million,  a  good  S&T  candidate. 
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Decreasing  the  time  required  for  sensing  the  target  missile  launch,  or  any  activity 
on  the  time  line  between  target  missile  launch  and  interceptor  launch,  is  worth  $30 
million  per  second,  a  possible  S&T  candidate.  Another  marginally  attractive  candidate  is 
decreasing  the  sensor  lag  in  the  hit-to-kill  vehicle,  which  is  worth  $  1 0  million  for  every 
10  millisecond  improvement. 

The  Technology  Value  Model  can  be  used  to  examine  new  intercept  methods, 
such  as  the  Explosively  Fonned  Projectile  (discussed  in  the  companion  volume  on  the 
Technology  Assessment  Pilot  Project).  The  Explosively  Fonned  Projectile,  is  an 
alternative  to  a  simple  hit-to-kill  vehicle,  expels  a  large  number  of  small  particles  at  very 
high  speed  toward  the  target.  The  Technology  Value  Model  shows  that  the  best 
Explosively  Formed  Projectile  system  is  superior  to  simple  hit-to-kill  when  the 
probability  of  hit  to  kill  destroying  the  target  is  less  than  30%.  Hit  to  kill  is  always 
superior  when  it  has  more  than  40%  chance  of  killing  the  target.  Between  30%  and  40%, 
the  results  depend  on  the  scenario. 

The  Explosively  Formed  Projectile  requires  a  smaller  amount  of  mass  to  actually 
hit  the  target  because  the  projected  particles  have  higher  velocity.  However,  only  a  small 
fraction  of  the  projected  mass  (at  best,  only  0.5%)  actually  strikes  the  target.  The  net 
result  is  that  system  requires  an  interceptor  payload  20  to  50  times  larger  than  the  simple 
kill  vehicle.  The  effect  on  the  size  of  the  interceptor  rocket  motor  and  the  size  of  the 
entire  deployed  BAAM  package  is  similar. 

Because  the  Explosively  Fonned  Projectile  makes  the  rocket  much  larger, 
technologies  that  reduce  rocket  size  become  much  more  important.  A  10-second  increase 
in  specific  impulse  is  worth  $300  million  in  an  Explosively  Formed  Projectile  system 
versus  $5  million  in  conventional  hit  to  kill.  Similarly,  a  1%  improvement  in  mass  ratio  is 
worth  $400  million  under  the  Explosively  Formed  Projectile  system  versus  $10  million 
for  the  baseline  system. 

In  summary,  the  Technology  Value  Model  provides  quantitative  insight  into  the 
relative  importance  of  different  avenues  of  S&T  investment,  as  well  as  the  ability  to 
evaluate  specific  technology  opportunities.  The  Buried  Autonomous  Anti-Missile 
Defense  System  Technology  Value  Model  is  a  prototype.  The  Technology  Value  Model 
can  be  extended  to  address  other  missile  defense  concepts.  With  application  in  practice  it 
can  also  be  matured  and  better  adapted  to  users  in  MDA/AS.  The  Technology  Value 
Model  can  be  extended  to  a  design  value  model,  an  objective  function  for  optimal 
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conceptual  design,  or  an  aid  to  evaluating  trade  studies.  It  can  form  the  basis  for  a 
distributed  optimal  design  approach  to  full  system  development. 

For  a  technology  value  model,  the  greatest  need  is  a  trustworthy  and  rigorous 
process  for  estimating  the  time,  cost,  and  risk  for  developing  specific  technologies. 
Research  into  historical  trends  of  progress  of  actual  technologies  through  technology 
readiness  levels  versus  time  and  expenditures  could  create  a  database  from  which 
technology  development  could  be  forecast  based  on  progress  through  readiness  levels 
and  spending  to  date.  Such  a  technology  development  forecasting  system  would  have 
wide  applicability. 

Conclusions  and  Next  Steps 

This  chapter  has  provided  an  overview  of  how  M&S  can  be  used  within  MDA/AS 
for  S&T  program  planning,  execution,  and  management,  and  it  has  provided  a  specific 
example  in  the  Missile  Defense  Technology  Value  Model.  To  increase  the  beneficial  use 
of  M&S  in  planning,  execution,  and  management  of  MDA’s  S&T  program,  we 
recommend  the  following  actions: 

1.  Establish  a  focal  point  within  MDA/AS  to  coordinate  relevant  M&S  activities. 

Such  a  focal  point  could  be  an  internal  individual  or  team  or  an  outside 

supplier  of  M&S  expertise. 

2.  Collect,  document,  and  evaluate  relevant  models  in  terms  of  their  applicability 

to  MDA/AS  needs.  Organizations  that  should  be  contacted  include  the 

following: 

•  Defense  Modeling  and  Simulation  Office. 

•  Service  M&S  offices. 

•  DoD  Modeling  and  Simulation  Information  Analysis  Center. 

•  Space  Users  Group  of  the  DoD  Modeling  and  Simulation  Analysis  Center. 

•  DoD  Modeling  and  Simulation  Resource  Repository. 

•  BMD  Simulation  Support  Center. 

•  BMDO  M&S  Resource  Repository. 

•  American  Institute  of  Aeronautics  and  Astronautics. 

•  Military  Operations  Research  Society. 

•  Simulation  Interoperability  Standards  Organization. 

•  Society  for  Computer  Simulation  International. 
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•  Other  organizations  involved  in  the  development  and  application  of 
relevant  models,  including  technology  developers,  FFRDCs  (e.g.,  IDA), 
and  organizations  such  as  the  Aerospace  Systems  Design  Laboratory  at 
Georgia  Tech. 

3.  Identify  M&S  voids  that  need  to  be  addressed.  This  could  be  done  by 
establishing  an  overall  M&S  architecture  or  conceptual  reference  model  for 
MDA  applications,  crating  specific  M&S  functionalities,  validation  and 
verification  of  new  or  existing  models,  and  coordination  among  ongoing  or 
planned  M&S  efforts  being  undertaken  by  other  organizations. 

4.  Fill  the  voids. 

5.  Facilitate  the  use  of  M&S  for  specific  S&T  applications  managed  by 
MDA/AS. 

6.  Consider  creating  an  integrated  MDA  conceptual  design  and  analysis  M&S 
environment  or  toolkit  and  related  infrastructure  for  use  in  S&T  program 
planning,  execution,  and  management. 

7.  Measure  the  impact  that  M&S  has  on  MDA/AS’s  S&T  program,  and  report 
this  impact  to  senior  MDA  and  DoD  management. 

8.  Extend  the  Technology  Value  Model  to 

•  Address  other  missile  defense  concepts  (with  application  in  practice  it  can 
also  be  matured  and  better  adapted  to  users  in  MDA/AS.) 

•  Serve  as  a  design  value  model,  that  is,  an  objective  function  for  optimal 
conceptual  design  or  an  aid  to  evaluating  trade  studies.  In  this  form,  it  can 
be  used  to  coordinate  distributed  design  teams. 

•  Incorporate  historical  information  on  the  time,  cost,  and  risk  for  developing 
specific  technologies  to  create  a  database  from  which  technology 
development  could  be  forecast  based  on  progress  through  readiness  levels 
and  spending  to  date. 

Modeling  and  simulation  technology  can  play  an  important  role  in  MDA/AS  S&T 
planning  and  management  by  reducing  research  cost  and  time  and  enabling  the 
exploration  of  more  advanced  concepts  and  emerging  technologies. 
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ACRONYMS 


ARL 

Army  Research  Laboratory 

AWSIM 

Air  Warfare  Simulation 

BAAM 

Buried  Autonomous  Anti-Missile  System 

BART 

Battle  Area  Regions  Threatened 

BMDS 

Ballistic  Missile  Defense  System 

C2W 

Command  and  Control  Warfare 

C4ISR 

Command,  Control,  Communications,  Computers, 
Intelligence,  Surveillance,  and  Radar 

CAPS 

Commander’s  Analysis  and  Planning  Simulation 

CATT 

C2W  Analysis  and  Targeting  Tool 

CECOM 

Communications  Electronics  Command 

CENTCOM 

Central  Command 

CEP 

Circular  Error  Probable 

COSMOS 

C4ISR,  Space  and  Missile  Operations  Simulation 

DARPA 

Defense  Advanced  Research  Projects  Agency 

DoD 

Department  of  Defense 

EADSIM 

Extended  Air  Defense  System 

FFRDC 

Federally  Funded  Research  and  Development  Center 

GIANT 

GPS  Interference  and  Navigation  Tool 

GLEEM 

Global  Positioning  System  End-to-End  Model 

GOTChA 

Goals,  Objectives,  Technical  Challenges,  Approaches 

GPS 

Global  Positioning  System 

IMAS 

Integrated  Modeling  and  Analysis  Suite 

INS 

Inertial  Navigation  System 

ITW/AA 

Integrated  Theater  Warning  and  Attack  Assessment 

JICM 

Joint  Integrated  Contingency  Model 

JSIMS 

Joint  Simulation  System 

JWARS 

Joint  Warfare  System 

GL-l 


LTAS 

Laser  Threat  Analysis  System 

M&S 

Modeling  and  Simulation 

MASC 

Model  for  Analysis  of  Sensor  Coverage 

MDA 

Missile  Defense  Agency 

MDA 

Missile  Defense  Agency 

MDA/AS 

Missile  Defense  Agency/Advanced  Systems  (Director) 

MDST 

Missile  Defense  Space  Tool 

MIP 

MDA  Innovation  Process 

MIRV 

Multiple  Independently  Targeted  Reentry  Vehicles 

NASA 

National  Aeronautics  and  Space  Administration 

NASM 

National  Air  and  Space  Model 

NUCSS 

USSPACECOM  Communications  Simulation  System 

OODA 

Observe,  Orient,  Decide,  and  Act  (loop) 

OSD 

Office  of  the  Secretary  of  Defense 

PSM 

Portable  Space  Model 

R&D 

Research  and  Development 

RDT&E 

Research,  Development,  Test,  and  Evaluation 

S&T 

Science  and  Technology 

SBMCS 

Space  Battle  Manager  Core  System 

SEAS 

System  Effectiveness  Analysis  Simulation 

SMAT 

Satellite  and  Missile  Analysis  Tool 

SPAAT 

Sensor  Platform  Allocation  Tool 

SPAWAR 

Space  and  Naval  Warfare  Systems  Command 

SST 

Spacecraft  Simulation  Toolkit 

STAMP 

Strategic  and  Theater  Attack  Modeling  Process 

STORM 

Strategic  and  Theater  Operations  Research  Model 

STK 

Satellite  Tool  Kit 

SWOT 

Strengths,  Weaknesses,  Opportunities,  Threats 

TRL 

Technology  Readiness  Level 

UAV 

Unmanned  Aerial  Vehicle 
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APPENDIX  A 

VIRTUAL  LABORATORY  CASE  STUDIES 

ROCKWELL  SCIENTIFIC  AND  BOEING  PHANTOM  WORKS 

In  1996,  Rockwell  International  Corporation  sold  most  of  its  defense  and 
aerospace  business  units  to  the  Boeing  Company.  At  that  time,  a  special  transition 
agreement  was  executed  between  the  two  corporations,  providing  for  continued  R&D 
support  to  Boeing  by  Rockwell’s  central  research  laboratory,  the  Rockwell  Science 
Center  (known  after  2001  as  Rockwell  Scientific  Company).  The  agreement  provided  for 
3  years’  guaranteed  funding  at  a  prescribed  level,  with  automatic  renewals  unless  Boeing 
gave  1  year’s  notice  of  cancellation.  The  collaboration  has  been  a  successful  one,  with 
both  sides  benefiting.  Rockwell  Scientific  has  enjoyed  a  stable  source  of  funding  and  a 
window  into  a  large  defense  business  and  its  government  customers.  Boeing  has  enjoyed 
access  to  world-class  research  capabilities  by  paying  a  relatively  low  fixed  fee,  without 
having  to  take  on  large  human  resource  commitments.  Perhaps  the  most  important 
testimony  to  the  collaboration’s  success  is  that  8  years  later,  at  the  time  of  this  writing, 
the  agreement  is  still  in  force. 

The  relationship  between  Boeing  and  Rockwell  Scientific  has  been  a  close  one, 
with  Rockwell  Scientific  acting  as  a  virtual  laboratory  division  of  Boeing’s  Phantom 
Works.  Technical  areas  of  collaboration  include  advanced  materials,  high-speed 
electronics,  high-data-rate  wireless  systems,  multimodal  integrated  displays,  and  various 
fields  of  infonnation  science.  Boeing  tends  to  use  Rockwell  Scientific  as  a  source  of 
unique  capabilities  rather  than  as  a  pool  of  extra  manpower  for  routine  tasks.  Were  this 
not  true,  the  relationship  would  probably  not  have  lasted. 

A  high-ranking  executive  at  Boeing  oversees  the  relationship,  and  Rockwell 
Scientific  has  a  full-time  Boeing  Program  Director.  Contacts  between  the  two 
organizations  are  both  broad  and  deep.  Division  directors  at  Rockwell  are  tightly  coupled 
to  executives  at  Phantom  Works,  as  are  individual  scientists  to  their  technical 
counterparts.  Projects  are  initiated  by  mutual  agreement  and  are  collaboratively  planned. 
Informal  reviews  on  most  projects  occur  weekly,  with  fonnal  reviews  occurring 
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quarterly.  Relationships  between  Rockwell  Scientific  and  Boeing  are,  on  some  occasions, 
closer  than  relationships  between  different  parts  of  the  Boeing  company  itself. 

Potential  conflicts  of  interest  do  arise.  On  occasion,  Rockwell  Scientific  and 
Boeing  Phantom  Works  may  compete  with  each  other  for  government  contracts,  although 
this  is  usually  avoided.  Of  more  concern  is  that  Rockwell  Scientific  has  a  close 
relationship  with  Rockwell  Collins  company’s  defense  division,  a  direct  competitor  with 
parts  of  Boeing’s  defense  businesses.  Further  complicating  the  relationship,  Boeing’s 
commercial  airplane  division  is  the  largest  customer  of  Rockwell  Collins’  commercial 
business,  which  in  turn  also  has  a  close  relationship  with  Rockwell  Scientific.  The  most 
difficult  conflicts  arise  when  Collins’  defense  business  competes  directly  with  Boeing  in 
a  government  procurement,  and  Rockwell  Scientific  is  required  to  support  both.  Such 
conflicts  are  managed  through  continual  vigilance  and  the  construction  of  strict  firewalls 
when  necessary.  The  agreement  between  Rockwell  Scientific  and  Boeing  specifies  that 
Boeing  owns  all  intellectual  property  arising  from  projects  it  funds.  Thus,  scientists  in  the 
same  research  group  at  Rockwell  Scientific  who  ordinarily  collaborate  closely  are 
enjoined  from  discussing  specific  projects  with  each  other  when  some  are  supporting 
Boeing  and  others  are  working  for  Collins. 

There  are  a  number  of  reasons  for  the  success  of  Rockwell  Scientific  as  a  virtual 
laboratory  division  of  Boeing  Phantom  Works,  the  most  important  of  which  are  the 
following: 

•  High-level  executive  attention  to  the  relationship  on  both  sides. 

•  Stable  multiyear  funding,  enabling  long-term  collaboration. 

•  The  existence  of  unique  capabilities  at  Rockwell  Scientific  that  augment  and 
complement  Boeing’s  internal  capabilities. 

•  A  proactive  approach  to  preventing  conflicts  of  interest. 

THE  ARMY  RESEARCH  LABORATORY  ADVANCED  DISPLAYS 
FEDERATED  LABORATORY 

An  example  of  a  long-term  virtual  laboratory  structure  created  by  an  agency  of 
the  United  States  Government  can  be  found  in  the  Army  Research  Laboratory  (ARL) 
Advanced  and  Interactive  Displays  Federated  Laboratory  (Displays  FedLab).1  A  $30M 
basic  research  effort  running  from  1996  to  2001,  its  goal  was  to  develop  new 


1  M.S.  Vassiliou,  “The  ARL  Displays  FedLab:  A  Partnership  between  Industry,  Government,  and 
Academia,”  Proc.  2000  IEEE  Aerospace  Conference,  2000,  pp.  521-529. 
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technologies  in  visualization,  collaborative  planning,  human-computer  interaction,  and 
related  areas. 

The  Displays  FedLab  began  as  a  consortium  led  by  Rockwell  Science  Center 
(now  known  as  Rockwell  Scientific),  then  a  major  corporate  research  laboratory.  The 
consortium  also  included  Rockwell's  Collins  business  unit,  a  leading  manufacturer  of 
commercial  and  military  avionics  systems;  the  University  of  Illinois  at  Urbana- 
Champaign’s  Bekmann  Institute,  an  internationally  renowned  center  of  excellence  in 
multimedia  and  visualization  technologies;  industrial  partners  MCNC  and  Sytronics;  and 
North  Carolina  Agricultural  and  Technical  College.  The  consortium  members  accepted 
central  direction  from  ARL  and  functioned  for  6  years  as  virtual  laboratory  divisions.  The 
Displays  FedLab  was  one  of  three  Federated  Laboratories  formed  by  ARL  in  1996,  the 
other  two  being  focused  on  advanced  sensors  and  advanced  telecommunications. 

The  FedLab  concept  was  arguably  the  most  important  of  a  series  of  management 
reforms  undertaken  by  ARL  in  the  mid  1990s,  in  response  to  tremendous  budget  and 
mission  pressures.  ARL  concentrates  on  basic  and  applied  research,  as  well  as  weapons 
analysis.  Its  primary  customers  are  the  research,  development  and  engineering  centers  of 
the  Army  Materiel  Command.  ARL  strives  to  achieve  preeminence  in  key  areas  of 
science  and  engineering  relevant  to  land  warfare;  to  foster  a  close  relationship  with  Army 
users  and  a  partnership  with  the  rest  of  the  defense  community;  and  to  interact 
extensively  with  academia,  industry,  and  other  government  laboratories.2  In  many  ways, 
ARL’s  role  is  similar  to  that  of  a  corporate  laboratory  in  a  large  company,  supporting  the 
various  R&D  and  advanced  development  centers  in  the  corporation’s  business  units  by 
creating,  importing,  evaluating,  and  adapting  technology.3 

Since  the  early  1990s  ARL  has  operated  under  extreme  pressure  on  its  resources. 
The  total  budget  dropped  from  nearly  $600M  in  FY1992  to  $356M  in  FY1998,  with  a 
concomitant  decrease  in  headcount.4  During  the  same  period,  the  Anny’s  senior 
leadership  assigned  ARL  an  entirely  new  and  very  substantial  mission.  After  Desert 
Storm,  then  Anny  Chief  of  Staff  General  Sullivan  directed  that  Anny  Materiel  Command 
develop  the  technology  necessary  to  “digitize  the  battlefield.”  This  directed  the  Army  to 


2  National  Academy  of  Sciences,  Army  Research  Laboratory  Technical  Assessment  Board:  1998 
Assessment  of  the  Army  Research  Laboratory,  Washington,  D.C.:  National  Academy  Press,  1999. 

3  E.A.  Brown,  Reinventing  Government  Research  and  Development:  A  Status  Report  on  Management 
Initiatives  and  Reinvention  Efforts  at  the  Army  Research  Laboratory ,  Report  ARL-SR-57,  Army 
Research  Laboratory,  1998. 

4  Ibid. 
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devise  systems  that  will  enable  real-time  situational  awareness  for 
battlefield  commanders  at  all  levels.  Such  awareness  demands  wireless, 
near-instantaneous  communication  vertically  and  horizontally,  with  total 
fusion  of  all  relevant  information  (intelligence,  weather,  and  terrain  data, 
logistics  information,  etc.),  which  must  then  be  displayed  in  a  readily 
comprehensible  format.5 

ARL  was  given  the  responsibility  of  conducting  the  basic  R&D  in  support  of  this  mission 
and  transferring  the  technology  to  the  Communications  Electronics  Command  (CECOM) 
Research  Development  and  Engineering  Center,  which  would  then  design  and  field  the 
systems. 

The  FedLab  concept  was  one  result  of  these  challenges  and  pressures.  By  joining 
in  a  long-tenn  partnership  with  several  industrial  and  academic  entities  to  attack  a  broad 
relevant  technology  area,  ARL  could  access  a  range  of  talent  that  would  be  practically 
impossible  to  grow  in  house.  Since  the  “digitization  of  the  battlefield”  required 
technology  development  in  areas  where  commercial  industry  and  certain  academic 
locations  were  already  very  advanced,  the  FedLab  could  serve  as  a  mechanism  for  ARL 
to  extract  significant  leverage. 

Although  in  the  Federated  Laboratory  structure  the  industrial  and  academic 
partners  received  funds  from  the  government,  the  arrangement  was  significantly  different 
from  the  usual  contractual  research  relationship.  The  primary  difference  was  that  the 
government  entity  in  question,  in  this  case  ARL,  was  not  a  simple  funding  agency  but  an 
active  participant  in  the  research,  collaborating  with  and  directing  its  virtual  divisions. 
The  FedLab  was  funded  by  a  relatively  new  government  instrument,  the  Cooperative 
Agreement.6  This  authority  was  granted  to  the  Department  of  Defense  under 
10USC2358.7  The  Cooperative  Agreement  allows  a  highly  collaborative  and  substantial 
relationship  between  the  government  and  the  vendor.  ARL  and  its  partners  could  jointly 
plan  and  execute  research,  jointly  report  results,  and  jointly  redirect  work  as  required. 
The  Cooperative  Agreement  should  not  be  confused  with  the  Cooperative  Research  and 
Development  Agreement,  a  different  instrument  created  by  the  Technology  Transfer  Act 
of  1986  to  facilitate  commercialization  of  technologies  developed  in  National 
Laboratories.8  There  is  no  relationship  between  these  two  types  of  arrangements. 


5  Ibid. 

6  Ibid. 

7  http  ://ww.darpa.mil/cmo/pdfs/23  5  8  .pdf. 

8  G.  Tassey,  The  Economics  of  R&D  Policy,  Quorum,  Westport,  Conn.,  1998. 
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The  Displays  FedLab  was  managed  by  a  committee  of  representatives  from  all 
members,  reporting  directly  to  a  program  manager  at  ARL.  The  management  committee 
and  ARL  jointly  prepared  annual  research  plans,  working  to  ensure  the  relevance  of  the 
research  to  customers  in  the  Army.  Great  care  was  taken  to  see  that  resources  were 
committed  to  the  technology  transition  process.  Although  the  program  was  funded  under 
Section  6.1  of  the  Defense  Research  and  Engineering  budget,  a  stream  of  Section  6.2 
transition  funds  was  also  separately  injected  to  aid  in  technology  transfer  to  ARL’s 
government  customers  (primarily  CECOM).  The  three  FedLabs  held  an  annual 
symposium  in  which  they  presented  reviewed  papers  and  demonstrations  highlighting  the 
previous  year’s  work.  In  addition,  the  FedLabs  were  reviewed  annually  by  the  Army 
Research  Laboratory  Technical  Assessment  Board,  composed  of  experts  from  the 
National  Research  Council.9 

From  a  technical  point  of  view,  the  Displays  FedLab  was  a  significant  success, 
delivering  new  technologies  in  collaborative  planning,  multimodal  integrated  display 
systems,  virtual  and  augmented  reality  systems,  and  many  others  to  the  Army  for  further 
development  and  in  some  cases  for  immediate  use.  However,  could  the  same  results  have 
been  achieved  through  simple  contracting  without  the  management  overhead?  Probably 
not:  simple  contracting  would  not  have  led  to  the  same  level  of  integration  in  the  systems 
ultimately  delivered  to  CECOM.  Was  the  FedLab  then  a  success  as  a  virtual  laboratory? 
The  answer  is  a  qualified  yes.  As  noted  above,  it  is  difficult  to  imagine  ARL  hiring  and 
maintaining  such  a  broad  range  of  world-class  talent  internally.  It  is  equally  difficult  to 
imagine  ARL  obtaining  the  same  level  of  sustained  collaboration  in  a  simple  contracting 
structure.  However,  the  Displays  FedLab  fell  short  of  the  ideal  of  a  virtual  laboratory  in  a 
number  of  ways: 

•  Funding  was  unstable  and  inconsistent;  budget  uncertainty  led  to  destructive 
conflicts  and  gaming  among  the  members. 

•  Centrifugal  forces  were  strong.  That  is,  individual  consortium  institutions  often 
shamelessly  pursued  their  own  divergent  interests  without  regard  for  the  goals 
of  the  enterprise  as  a  whole.  Some  of  them  occasionally  did  “end  runs”  around 
the  consortium  and  attempted  to  market  their  own  activities  to  ARL  as  if  it 


9  National  Academy  of  Sciences,  Army  Research  Laboratory’  Technical  Assessment  Board:  1997 
Assessment  of  the  Army  Research  Laboratory,  Washington,  D.C.:  National  Academy  Press,  1998; 
Also  NAS  (1999) 
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were  simply  a  contract  monitor.  They  were  sometimes  successful  in  these 
efforts,  distorting  the  consortium’s  integrated  plans. 

•  Most  research  staff  members  at  Consortium  institutions  were  not  working  full 
time  on  the  program  and  thus  did  not  fully  “report  into”  ARL  as  members  of  a 
true  virtual  division. 

•  Staff  rotation  policies  from  the  consortium  institutions  to  ARL  and  vice  versa 
were  not  followed  to  the  degree  originally  envisioned  (in  the  end,  no  one  could 
be  forced).  The  researchers  in  the  various  member  institutions  did  not  interact 
as  much  as  they  could  have.  More  important,  not  enough  of  their  knowledge 
was  captured  and  used  by  ARL. 

•  In  some  cases,  ARL  did  not  have  enough  internal  capability  to  engage  fully 
with  the  virtual  divisions,  leading  to  some  working  independently.  The  reason 
for  having  the  virtual  divisions  in  the  first  place  was  so  that  ARL  could  draw 
on  capabilities  it  did  not  have;  however,  without  some  minimal  dedicated 
capability,  ARL  often  could  not  successfully  interact  with  the  virtual  divisions. 

Overall,  though,  the  FedLabs  can  probably  be  counted  as  a  successful  experiment. 
The  Displays  FedLab,  along  with  its  two  sister  consortia  in  sensors  and 
telecommunications,  received  the  “Hammer  Award”  in  1998  from  the  U.S.  Vice 
President’s  National  Partnership  for  Reinventing  Government,  as  a  new,  effective,  and 
efficient  way  for  the  Government  to  fund  and  participate  in  R&D.  In  2001,  a  new  round 
of  multiyear  Federated  Laboratories  (renamed  “Collaborative  Technology  Alliances”) 
was  funded. 


A-6 


APPENDIX  B 

NANOTECHNOLOGY  NETWORKING  OPPORTUNITIES 


The  field  of  nanotechnology — the  process  of  manipulating  matter  at  the  atomic 
scale — has  expanded  greatly  over  the  past  few  years  and  has  the  potential  of  providing 
significant  technological  benefit.  Current  government  programs  in  nanotechnology  that 
constitute  the  National  Nanotechnology  Initiative  are  being  coordinated  through  the 
Nanoscale  Science,  Engineering  and  Technology  Subcommittee  of  the  National  Science 
and  Technology  Council.  The  Council  was  established  in  1993.  It  is  a  cabinet-level 
council  that  coordinates  S&T  policies  and  programs  across  the  government  and  attempts 
to  establish  national  goals  for  federal  S&T  investments. 

Member  organizations  of  the  Nanoscale  Science,  Engineering  and  Technology 
subcommittee  are  listed  below: 

Chair:  M.C.  Roco,  National  Science  Foundation 
Executive  Secretary:  J.S.  Murday,  Naval  Research  Laboratory 

Members: 

Department  of  Agriculture 

Department  of  Commerce 

Department  of  Defense 

Department  of  Energy 

Department  of  Homeland  Security 

Department  of  Justice 

Department  of  State 

Department  of  Transportation 

Department  of  the  Treasury 

Environmental  Protection  Agency 

Food  and  Drug  Administration 

Intelligence  Community 

National  Aeronautics  and  Space  Administration 

National  Institutes  of  Health 

National  Institute  of  Standards  and  Technology 

National  Science  Foundation 

Nuclear  Regulatory  Commission 

Office  of  Management  and  Budget 

Office  of  Science  and  Technology  Policy 
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Table  B-l  shows  federal  funding  for  nanotechnology  over  the  past  3  years  FY01 
and  FY02  funds  are  actual  expenditures,  the  funding  for  FY03  are  estimates,  and  FY04 
are  budget  requests.  Of  the  total  funds  shown  in  FY03,  $32  million  ($29  million  in 
NASA’s  total  and  $3  million  in  the  U.S.  Department  of  Agriculture)  are  in  associated 
programs.  The  rest  of  the  funding  is  directly  in  support  of  nanotechnology  development. 
All  funding  shown  is  in  millions  of  dollars. 

Table  B-1.  Federal  Funding  for  Nanotechnology 


Agency 

FY01 

FY02 

FY03 

FY04 

Department  of  Defense 

163 

227 

243 

222 

Department  of  Energy 

88 

91 

139 

197 

Department  of  Justice 

1 

1 

1 

1 

Department  of  Transportation 

0 

2 

2 

2 

Environmental  Protection  Agency 

5 

5 

5 

5 

National  Aeronautics  and  Space  Administration 

22 

46 

51 

31 

National  Institutes  of  Health 

40 

41 

43 

70 

National  Institute  of  Standards  and  Technology 

33 

38 

44 

62 

National  Science  Foundaion 

150 

199 

221 

249 

Department  of  Agriculture 

2 

2 

3 

10 

Total  Funding 

504 

652 

752 

850 

Table  B-2  breaks  down  DoD  funding  by  organization  and  funding  category.  Again,  the 
funds  shown  are  in  millions  of  dollars.  The  FY01  and  FY02  funds  are  actual 


expenditures,  the  FY03  funds  are  estimates,  and  the  FY04  figures  are  budget  requests. 


Table  B-2.  DoD  Funding  for  Nanotechnology 


Organization 

FY01 

6.1  6.2+ 

FY02 

6.1  6.2+ 

FY03 

6.1  6.2+ 

FY04 

6.1  6.2+ 

DDR&E/DUS 

n 

30 

0 

26 

0 

28 

0 

28 

0 

u 

DARPA 

34 

21 

52 

62 

49 

93 

36 

81 

Army 

8 

1 

18 

9 

18 

11 

20 

10 

Air  Force 

8 

10 

11 

9 

9 

4 

12 

6 

Navy 

46 

5 

38 

2 

30 

1 

28 

1 

Total 

126 

37 

145 

82 

134 

109 

114 

108 

A  significant  portion  of  the  6.1  program  in  OSD  is  part  of  a  Defense  University 
research  initiative  on  nanotechnology.  New  DARPA  programs  include  nanostructures  in 
biology,  quantum  infonnation,  and  molecular  electronics.  The  Air  Force’s  basic  research 
activities  are  in  nanocomposites;  self-assembly  and  nanoscale  processing  for  the 
realization  of  3-D  optical  and  electronic  circuitry;  highly  efficient  space  solar  cells; 
nanoenergetics;  nanostructures  for  highly  selective  sensors;  nanoelectronics, 
nanomagnetics,  nanophotonics;  and  nanostructured  coatings,  ceramics,  and  metals.  The 
Anny  is  allocating  $10M  of  the  basic  research  funds  for  a  University  Affiliated  Research 
Center — Institute  for  Soldier  Nanotechnologies.  The  purpose  of  this  center  is  to  develop 
nanometer-scale  science  and  technology  solutions  for  the  soldier.  The  center  will 
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emphasize  revolutionary  materials  research  to  improve  advanced  soldier  protection  and 
survivability  capabilities.  The  Naval  Research  Laboratory  is  establishing  a  nanoscience 
institute.  Scheduled  to  open  this  summer,  the  institute  will  enhance  multidisciplinary 
thinking  and  critical  infrastructure.  A  recent  thrust  in  nanotechnology  research  is  to 
counter  the  current  terrorist  threat.  Nanoscience  is  showing  great  promise  for  an  array  of 
inexpensive,  integrated,  miniaturized  sensors  for  chemical,  biological,  radiological,  and 
explosive  agents;  protection  against  agents;  and  nanostructures  that  neutralize  agents. 

Government-sponsored  investments  in  nanotechnology  are  not  limited  to  the 
United  States  Worldwide  these  investments  totaled  approximately  $3  billion  in  FY03.  In 
addition  to  the  $700  million  in  the  United  States,  Japan  is  investing  $800  million, 
Western  Europe  $600  million,  and  other  countries,  including  China  and  Korea,  an 
additional  $750  million.  Information  about  industry  investments  in  the  United  States  is 
difficult  to  acquire  because  they  are  considered  competition  sensitive.  The  best  estimates 
are  amounts  roughly  equivalent  to  the  federal  investment. 

Currently,  20  government-sponsored  centers  in  the  U.S.  have  a  nanotechnology 
focus;  all  are  associated  with  universities: 
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National  Science  Foundation 

Nanoscale  Science  &  Engineering  Centers 

Nanoscale  Systems  in  Infonnation  Technology 
Nanoscience  in  Biological  &  Environmental  Engineering 
Integrated  Nanopattering  &  Detection 
Electronic  Transport  in  Molecular  Nanostructures 
Science  of  Nanoscale  Systems  &  Device  Applications 
Directed  Assemble  of  Nanostructures 

Nanobiotechnology,  Science  &  Technology  Center 

Material  Research  Science  &  Engineering  Centers 

Nanoscopic  Materials  Design 

Nanostructures  Materials 

Semiconductor  Physics  in  Nanostructures 

Nanostructured  Materials  in  Interfaces 

Quantum  &  Spin  Phenomena  in  Nanomagnetic  Structures 

Research  on  the  Structure  of  Matter 

Department  of  Defense 

Institute  for  Soldier  Nanotechnologies 
Center  for  Nanoscience  Innovation  for  Defense 
Nanoscience  Institute 


Cornell  University 
Rice  University 
Northwestern  University 
Columbia  University 
Harvard  University 
Rensselaer  Poly  Institute 

Cornell  University 


University  of  Virginia 
UC  Santa  Barbara 
Univ  Oklahoma  &  Arkansas 
Univ  Wisconsin  Madison 
Univ  Nebraska  Lincoln 
Univ  Pennsylvania 


MIT 

UC  Santa  Barbara 
Naval  Research  Laboratory 


NASA 

Institute  for  Cell  Mimetic  Space  Exploration 
Institute  for  Intelligent  Bio-Nanomaterials 
&  Structures  for  Aerospace  Vehicles 
Bio-Inspection,  Design  and  Processing  of 
Multi-functional  Nanocomposites 
Institute  for  Nanoelectronics  &  Computing 


UCLA 
Texas  A&M 

Princeton 

Purdue 


Additional  centers  are  planned.  For  example,  the  DOE  will  be  funding  the 
establishment  of  five  centers  valued  at  approximately  $85  million  apiece  in 
nanotechnology. 

Given  this  interest  and  potential  opportunities,  one  question  remains:  Will 
nanotechnology  help  MDA  meet  its  mission?  Discussions  with  Dr.  Murday  of  the  Naval 
Research  Laboratory  and  Executive  Secretary  of  the  National  Nanotechnology  Initiative 
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provided  some  insight  into  the  following  areas  of  nanotechnology  research  that  will  have 
an  impact  on  MDA. 

Electronics — Microelectronics  is  driven  by  miniaturization  which  leads  to 
increased  performance.  The  process  of  miniaturization  is  characterized  by 
Moore’s  law,  which  says  that  the  number  of  transistors  on  a  chip  doubles  every 
18  months.  Many  predicted  ends  to  Moore’s  law  have  come  and  gone,  but  it 
appears  that  the  industry  will  be  reaching  its  limits  in  silicon-based  devices  over 
the  next  10  years.  Nanotechnology  and  molecular  electronics  may  provide  the 
ability  to  manufacture  devices  that  have  dimensions  of  10  mn  and  lower.  This  is 
the  focus  of  a  DARPA’s  Molecular  Electronics  program.  Smaller  and  faster 
electronics  will  lead  to  smaller  and  faster  systems  that  are  and  will  remain  of  vital 
importance  to  MDA. 

Advanced  Materials — Carbon  nanotubes — sheets  of  graphite  rolled  up  with  their 
edges  connected  to  form  a  cylinder — hold  the  promise  of  extremely  high  tensile 
strength  inclusions  for  nanocomposites,  structural  beams  for  nanomachines,  and 
possibly  conductors  for  nanoelectronics.1  This  will  lead  to  significant 
improvements  in  strength-to-weight  ratios  of  structural  materials.  In  addition, 
materials  research  will  lead  to  improved  sensors,  energetic  materials,  batteries, 
solar  cells,  and  fuel  cells.  All  of  these  technologies  will  be  of  vital  importance  to 
MDA. 

One  can  quickly  reach  the  conclusion  that  MDA  should  be  involved  in 
nanotechnology  research.  An  initial  step  would  be  to  start  to  interact  with  the  current 
activities  underway  to  coordinate  the  various  government  programs.  The  first  of  these  is 
the  Nanoscale  Science,  Engineering  and  Technology  Subcommittee  of  the  National 
Science  and  Technology  Council.  As  discussed  above,  this  is  an  interagency  working 
group  whose  mission  is  to  coordinate  government  research  in  nanotechnology.  Key 
government  players  of  the  National  Nanotechnology  Initiative  that  meet  monthly  include: 

•  Dr.  Mihail  Roco  and  Dr.  Thomas  Weber  from  the  National  Science  Foundation 

•  Dr.  J.S.  Murday  from  the  Naval  Research  Laboratory 

•  Dr.  Clayton  Teaque  from  the  Department  of  Commerce 

•  Dr.  Gemot  S.  Pomrenke  form  the  Air  Force’s  Office  of  Scientific  Research 

•  Dr.  Clifford  Lau  from  the  Office  of  the  Secretary  of  Defense 

•  Dr.  Richard  Lareau  from  Homeland  Security 


1  “Materials  Engineering  with  Nature’s  Building  Blocks,”  AMPTIAC  Quarterly,  Volume  6,  Number  1, 
p.  31. 
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•  Dr.  Minoo  Dastoor  from  NASA 

Legislation  authorizing  the  National  Nanotechnology  Initiative  is  working  its  way 
through  Congress.  As  recommended  by  an  NRC  report,  the  legislation  will  call  for  an 
external  panel  to  review  the  initiative.  A  President’s  Council  of  Advisors  on  Science  and 
Technology  panel  is  being  fonned  to  accomplish  that  review.  In  addition,  this  year’s 
Defense  Authorization  Act  has  required  that  DoD  develop  and  submit  to  Congress  a 
strategic  plan  for  nanotechnology  by  March  2004.  Fiscal  year  2004  language  will  be 
requiring  the  same  from  the  other  agencies.  Dr.  Cliff  Lau  from  OSD  is  currently 
revitalizing  DoD  working  groups  to  develop  the  Department’s  strategic  plan  in 
nanotechnology,  which  will  have  to  be  coordinated  with  the  other  agencies. 

A  series  of  “Grand  Challenge”  workshops  are  underway  and  will  continue 
through  next  January  in  support  of  these  efforts.  This  process  and  the  development  of  a 
DoD  Strategy  would  provide  MDA  an  excellent  opportunity  to  become  engaged.  It  is  my 
understanding  that  MDA  was  invited  to  participate  in  these  DoD  and  interagency 
activities  when  they  were  initiated  about  5  years  ago,  but  declined. 

There  is  reason  and  opportunity  for  MDA  to  become  actively  involved  in  the 
current  nanotechnology  research  underway  in  DoD  and  the  other  government  agencies.  It 
is  considerably  more  difficult  to  determine  what  should  be  the  extent  of  that  involvement. 
To  answer  this  question  MDA  will  have  to  decide  what  areas  of  nanotechnology  provide 
the  greatest  opportunities  to  help  meet  its  mission.  Once  these  have  been  identified,  the 
level  of  existing  funding  already  committed,  organizations  performing  research  in  these 
areas,  and  the  direction  of  the  existing  research  can  be  used  to  determine  MDA’s  level  of 
participation  and  with  whom  they  need  coordinate. 

Suggested  next  steps: 

1 .  Continue  to  gather  information  on  nanotechnology  and  research  underway. 

2.  Become  involved  in  the  activities  underway  to  develop  a  strategic  plan  for 
DoD’s  nanotechnology  research  and  possible  the  National  Science  and 
Technology  Council  activities. 

3.  With  Dr.  Murday  from  the  Naval  Research  Laboratory,  prepare  a  briefing 
outlining  current  levels  of  research,  areas  that  will  affect  MDA,  and 
opportunities  for  cooperation  within  DoD  and  the  other  agencies.  This 
briefing  could  be  provided  to  Gary  Payton. 
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APPENDIX  D 

MISSILE-RELATED  SIMULATION  MODELS1 


Model  Name 

Description 

Point  of  Contact 

Battle  Area  Regions 
Threatened  (BART) 

BART  is  a  PC-based  model  used  to  conduct  analyses  of 
ballistic  missile  defense  architectures.  It  models  radars, 
threat  missiles,  interceptor  missiles,  and  space-based 
warning.  It  outputs  data  on,  among  other  things, 
engagement  opportunities,  kill  assessment  opportunities, 
and  intercept  angles.  BART  allows  an  analyst  to  design 
an  arbitrary  defensive  architecture  composed  of  ground- 
based  radar  sensors  and  ground-based  ballistic 
interceptor  missiles.  The  performance  parameters  of  the 
radars  and  interceptor  missiles  as  well  as  their  locations 
are  all  user  defined.  BART  has  the  capability  to  simulate 
the  early  warning  afforded  by  space-based  launch 
detection  sensors  such  as  the  Defense  Support  Program 
(DSP)  or  Space  Based  Infra  Red  System  satellites. 

NORAD 

Ms.  Isabelle  Julien 

719-554-3781 

isa  belle.  iulien(S)peterso 

n. af.mil 

Commander’s 

Analysis  and 

Planning  Simulation 
(CAPS) 

CAPS  is  used  to  solve  problems  involving  Theater  Air 
and  Missile  Defense  and  Ballistic  Missile  Defense.  It 
covers  the  most  common  missile  types,  threats  and 
friendly  defenses  as  well  as  the  Airborne  Laser,  Ground 
Based  Interceptors  and  the  Aegis  LEAP  Interceptor.  It  is 
PC  based. 

SPARTA  Inc., 

Mr.  Dave  Eissler 

CAPS  Program  Office 
703-797-3068 

caps.questions(5)sparta. 

com. 

Extended  Air 

Defense  System 
(EADSIM) 

EADSIM  is  a  system-level  simulation  used  by  combat 
developers,  materiel  developers  and  operational 
commanders  to  assess  the  effectiveness  of  theater 
missile  defense  and  air-defense  systems  against  the  full 
spectrum  of  extended  air-defense  threats.  EADSIM 
models  fixed-  and  rotary-wing  aircraft,  tactical  ballistic 
missiles,  cruise  missiles,  infrared  and  radar  sensors, 
satellites,  command-and-control  structures,  sensor  and 
communications  jammers,  communications  networks 
and  devices,  and  fire  support  in  a  dynamic  environment 
that  includes  the  effects  of  terrain  and  attrition  on  the 
outcome  of  the  battle.  The  tool  provides  analysts  and 
training  audiences  insights  into  theater  missile  defense 
architecture,  battle  management,  system  employment  for 
maximum  effectiveness,  force  structure  analysis,  and 
mission  planning. 

Army  Space  and  Missile 
Defense  Command 

Mr.  Jim  Watkins, 
iim.watkins(S)smdc.armv 

.mil 

256-955-1681 

or  Mr.  Page  Stanley 
Teledyne-Brown 
Engineering  (TBE) 

paqe.stanlev(S)tbe.com 

256-726-1866  or 
www.eadsim.com 

1  “Space,  Missile  Defense  &  Information  Operations  Models  and  Simulations  Catalog  (for 
USSTRATCOM),”  26  August  2002,  (http://www.msiac.dmso.mil/spug  documents/ 
spacecatalog26aug.doc). 
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Model  Name 

Description 

Point  of  Contact 

GPS  Interference  and 
Navigation  Tool 
(GIANT) 

GIANT  is  a  constructive  and  repeatable 
engagement/mission-level  simulation  that  calculates  the 
impact  of  navigation  performance  on  warfighter 
measures  of  effectiveness  (e.g.,  target  kills).  A  GIANT 
scenario  consists  of  a  GPS/INS-equipped  platform 
moving  over  digital  terrain  (i.e. ,  digital  terrain  elevation 
data),  on  a  WGS-84  Earth,  under  a  moving  GPS 
constellation  transmitting  multiple  codes  on  multiple 
frequencies.  GIANT  can  represent  any  air  or  ground 
vehicle  with  or  without  weapons.  Weapons  also  have 
GPS/INS  navigation  systems  and  the  launch  platform  to 
weapon  handoff  event  is  modeled.  As  an  option,  any 
number  of  stationary  or  moving  GPS  jammers  can  be 
present.  Target  miss  distance  and  probability  of  kill  is 
thus  traceable  to  the  weapon  and  the  launcher. 

SMC/CZE 

1LT  Amon  Dothard 

amon.dothard(5)losanqe 

les.af.mil 

310-363-3124 

or  Mr.  Steve  Friedman 

Veridian  Engineering 

www.veridian.com  937- 
476-2509 

Joint  Warfare  System 
(JWARS) 

JWARS  is  under  development  to  be  a  state-of-the-art, 
constructive  simulation  that  provides  a  multisided  and 
balanced  representation  of  joint  theater  warfare  that  is 
capable  of  use  in  analysis  of  planning  and  execution, 
force  assessment,  system  effectiveness  and  trade-off 
analysis,  and  concept  and  doctrine  development  and 
assessment.  It  will  be  a  balanced  warfare  representation 
including  C4,  ISR  and  logistics  and  will  focus  on  the 
operational  level  of  war.  It  will  replace  MIDAS  and 
TACWAR. 

Mr.  Don  Bates 

JWARS  Joint  Program 
Office 

703-696-9490 

or  JWARS  Help  Desk 

913-684-8080  or 
https://www.iointmodels. 

smil.mil 

Missile  Defense 

Space  Tool  (MDST), 
formerly  Portable 
Space  Model  (PSM) 

MDST  provides  the  capability  to  support  live  or  simulated 
exercises  by  injecting  missile  warning  message  sets  into 
operational  communications  and  simulation  networks. 
MDST  contains  real-time  models  designed  to  provide  a 
representation  of  the  Defense  Support  Program,  the 
Satellite  Based  Infrared  System,  and  elements  of  the 
Theater  Event  System  at  a  sufficient  level  of  fidelity  to 
support  exercises  while  operating  in  real  time.  It  notifies 
theater  commanders  of  theater  ballistic  missile  launches 
via  the  Tactical  Information  Broadcast  System  and  the 
Tactical  Related  Applications  Data  Dissemination 

System. 

JNIC 

MAJ  Dave  Silvernail 

719-567-9373/0766 

Lightning 

Lightning  is  a  strike-campaign-level  simulation  used  to 
support  analyses  of  the  operational  contribution  of  space 
systems  (e.g.,  ISR,  navigation,  weather, 
communications)  using  operational  metrics  such  as 
targets  destroyed  over  time,  time  to  achieve  operational 
phase  goals,  time  to  halt,  and  attrition  over  time. 

Lightning  is  an  aggregated  model  that  supports  quick 
turnaround  studies  with  very  rapid  execution,  rapid 
modification,  and  rapid  database  development. 

North  rup-Grumman/IT- 
TASC  Lightning 

Solutions 

Dr.  Gregg  Burgess 

qburqess(5)northrupqru 

mman.com 

703-793-3700  x2310 
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Model  Name 

Description 

Point  of  Contact 

National  Air  &  Space 
Model  (NASM) 

NASM  is  the  Air  Force  component  of  the  Joint  Simulation 
System  (JSIMS).  It  is  the  successor  to  the  Air  Warfare 
Simulation  (AWSIM).  NASM  is  developing  the  mission 
space  objects  (systems,  organizations  and  procedures) 
JSIMS  will  use  to  provide  the  functional  capability  to 
represent  the  full  range  of  aerospace  power  applications 
in  a  joint  synthetic  battlespace  for  both  Air  Force  specific 
and  joint  training.  Applications  include  training  and 
readiness,  education,  doctrine  development,  situation 
assessment,  and  the  formulation,  assessment,  and 
rehearsal  of  operational  plans.  The  IOC  version  of 

NASM  will  likely  include  a  limited  depiction  of  all 
satellites  (basic  orbital  characteristics)  and  higher  fidelity 
models  of  missile  warning  (Defense  Support  Program  & 
Space  Based  Infra  Red  System),  navigation  (GPS), 
some  satellite  communications,  and  foreign  space 
control. 

LtCol  Emily  Andrew 

AFMC  ESC/CXC 

781-377-6421 

emilv.andrew(5)hansco 

rn.af.mil 

or  Mr.  Paul  Driscoll 

781-377-2670 

Paul.driscoll(5)hanscom. 

af.mil 

Satellite  &  Missile 
Analysis  Tool  (SMAT) 

SMAT  is  a  comprehensive  two-dimensional  and 
three-dimensional  animated  visual  modeling  tool  for 
analysis  of  orbiting  bodies,  ballistic  missile  trajectories, 
and  their  relationship  to  Earth.  SMAT  provides  a  fully 
modeled  Earth  with  detailed  geographic  and  political 
boundaries,  has  the  capability  to  zoom  and  rotate  the 
viewing  position  of  Earth,  and  provides  accurate  Sun 
position  and  illumination.  Databases  within  SMAT 
contain  the  parameters  for  the  Tactical  Warning/Attack 
Assessment  System,  the  Air  Force  Satellite  Control 
Network,  and  the  Space  Surveillance  Network  sensors. 
SMAT  allows  complete  control  of  all  displayed  sensor 
parameters,  both  ground  and  space-based,  and  allows 
importing,  editing  and  saving  of  additional  sensor 
parameters.  SMAT  provides  the  capability  to  model 
ballistic  missile  launch  profiles,  both  strategic  and 
theater,  from  any  point  on  the  surface  of  Earth. 

Ms.  Kathy  Gue 

USAF/SWC/DOG 
(Space  Warfare  Center) 
kathleen.que(S)swc.schri 

ever.af.mil 

719-567-9289 
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Model  Name 

Description 

Point  of  Contact 

Satellite  Tool  Kit 
(STK) 

STK  4.0  (basic)  is  a  free  commercial  off-the-shelf 
product  that  provides  sophisticated  modeling  functions 
for  space-  and  ground-based  objects,  such  as  satellites, 
ships,  aircraft,  and  land  vehicles.  Functions  included  in 
the  free  version  of  the  software  include  vehicle 
propagation,  determining  visibility  areas  and  times,  and 
computing  sensor  pointing  angles.  Free  STK  provides 
animation  capabilities  and  a  two-dimensional  map 
background  for  visualizing  the  paths  of  vehicles  over 
time.  Results  can  be  generated  in  both  textual  and 
graphical  formats.  Additional  modules  can  be  purchased 
to  provide  enhanced  computational  and  visualization 
capabilities.  In  particular,  STK’s  Visualization  Option 
provides  dynamic  three-dimensional  display  of  STK 
scenarios.  A  host  of  additional  modules  are  available  to 
provide  detailed  analyses  for  such  tasks  as  determining 
satellite  coverage  overtime,  visibility-related  access  for 
networks  of  objects,  rapid  analysis  of  close  encounters 
between  orbiting  objects,  realistic  missile  flight  modeling, 
are  detailed  modeling  of  radar  systems  and  satellite 
communications  link  analysis.  It  addresses  mission 
planning,  launch,  and  ballistic  missile  flight.  STK  is  used 
to  examine  alternative  deployments  of  satellites  within 
constellations  and  analyze  alternative  coverage  of 
combinations  of  satellites. 

Analytical  Graphics,  Inc. 

Mrs.  Tina  Cox 

719-573-2600 

or  1-800-220-4STK 

tcox(5)stk.com  or 
www.stk.com 

Strategic  and  Theater 
Attack  Modeling 
Process  (STAMP) 

STAMP  is  a  ballistic-missile  and  space-launch-vehicle 
flight  generator  and  engineering  analysis  tool.  It  can 
model  missile  flights  from  launch  to  impact  and  present 
extensive  flight  characteristics  and  trajectory 
descriptions  using  a  wide  array  of  graphical  and  tabular 
outputs.  STAMP  can  also  model  numerous  U.S.  and 
foreign  space  launch  vehicles  from  launch  to  orbital 
insertion.  STAMP  features  an  easy-to-use  operator 
interface  using  windows  and  click-type  menu  selections. 
STAMP  is  driven  by  detailed  engineering  databases, 
developed  and  approved  by  the  appropriate  intelligence 
agencies,  that  have  the  parameters  and  values  needed 
to  model  strategic  and  theater  missiles  as  well  as  foreign 
space  launch  vehicles  consistent  with  intelligence 
estimates.  Portions  of  STAMP  have  been  integrated  into 
the  SMAT  to  generate  and  process  foreign  missile 
trajectories  for  SMAT  users.  STAMP  was  developed  by 
SAIC  under  the  sponsorship  of  the  Air  Force  National  Air 
Intelligence  Center. 

1LT  Tempalski 

937-257-2356 

rntl  81  (®wpafb. af.mil  or 
George  Panson 

937-257-2356 

qmp268(S)wpafb.af.mil 

Command  and 

Control  Warfare 
(C2W)  Analysis  and 
Targeting  Tool 
(CATT) 

CATT  provides  a  simulation  capability  of  an  adversary's 
integrated  air  defense  system  and  the  capability  for 
analysts  to  do  sensitivity  analysis  on  alternative  actions. 

It  includes  end-to-end  modeling  of  integrated  air  defense 
system  processes  such  as  detection,  tracking,  weapons 
allocation,  communication,  decision-making  and 
engagement.  The  model's  primary  C2W  actions  include 
inserting  and  removing  various  user-defined  flight  paths 
and  removing  various  communications  links  and  radar 
posts. 

AFIWC/SAA 

LtCol  Ross  Ziegenhorn 

razieqe(5)afiwc.osis.qov 

210-977-2427 
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C4ISR,  Space  and 
Missile  Operations 
Simulation 
(COSMOS) 

COSMOS  has  been  developed  to  support  analysis  of  the 
performance  of  C4ISR,  space,  and  missile  systems. 
COSMOS  explicitly  models  collection  systems  for 

SIGINT,  IMINT,  and  HUMINT  as  well  as  surveillance 
systems  using  visible,  IR,  LADAR  and  RADAR 
technologies.  The  resources  and  associated  time  lines 
required  to  process,  exploit,  and  disseminate  the 
collected  information  are  modeled  using  a  flexible  rule- 
based  approach.  COSMOS  can  also  model  systems  in 
all  Space  Command  mission  areas  including  futuristic 

U.S.  and  foreign  space  control  systems  such  as  space- 
based  lasers,  ground-based  lasers  and  kinetic  energy 
antisattelite.  The  model  is  currently  in  use  supporting 

OSD,  Joint  Staff,  Air  Force,  Army,  Navy,  Marine,  Office 
of  the  Space  Architect,  and  classified  customer 
analyses.  COSMOS  has  been  interfaced  with 
community-accepted  classified  models  to  support 
analysis  of  current  and  future  system  architecture 
performance.  COSMOS  was  developed  and  continues  to 
be  enhanced  by  the  SAIC  Pentagon  On  Site  Team. 

Jeff  Knox 

SAIC 

703-276-2116 

JEFFREY.S.KNOXOsai 

c.com.) 

EDGE  Developer 
Option 

EDGE  Developer  Option  is  a  commercial  off-the-shelf 
set  of  visualization,  simulation,  and  analysis  tools  and 
applications  that  allow  users  to  create  a  rich  synthetic 
environment  and  view  the  world  from  outer  space  to  sea 
level.  The  foundation  is  the  EDO  Visualization 

Component.  Additional  components  provide  libraries  for 
integrating  imagery,  maps,  terrain,  time,  and  weather. 

The  Ascent  tool  for  launch  vehicle  trajectory  simulation 
component  can  also  run  as  a  stand-alone  program. 

http://www.autometric.c 

om 

Extended  Air 

Defense  Testbed 
(EADTB) 

EADTB  allows  the  analyst  to  model  a  broad  range  of 
military  missile  defense  applications  from  the  fire  unit 
level  to  the  theater  level  in  a  constructive  simulation 
framework.  Its  object-based  simulation  architecture 
supports  this  range  by  allowing  the  user  to  develop 
system  models  called  specific  system  representations. 

The  user/analyst  can  place  numbers  of  these  tailored 
simulated  systems  on  a  host  gameboard  without  having 
to  rewrite  other  existing  system  models  or  modify  the 
supporting  architecture.  A  major  strength  of  EADTB  is 
the  capability  to  model  the  BMC4I  to  the  level  necessary 
to  answer  complicated  joint  service  interoperability 
issues.  EADTB  has  obtained  Defense  Information 

Service  compatibility  and  is  pursuing  High  Level 
Architecture  compliance  at  this  time. 

Army  Space  and  Missile 
Defense  Command 
(SMDC)  Battle  Lab 

Mr.  Robert  Karl 

256-955-1685 

robert.karl(a)smdc.armv. 

mil 

Global  Positioning 
System  End-to-End 
Model  (GLEEM) 

GLEEM  was  developed  to  assist  in  analysis  of 
capabilities  and  vulnerabilities  of  Global  Positioning 
Systems  (GPS)  and  Inertial  Navigation  Systems  (INS)  in 
aircraft  and  guided  munitions.  GLEEM  allows  projection 
of  GPS  receiver  performance  in  signal-lock  maintenance 
while  in  a  hostile  or  benign  environment  and  simulates 
various  combinations  of  antennas  and  receivers,  on 
multiple  platforms,  with  multiple  jammers.  Friendly 
interference  platforms  can  be  included  as  well. 

AFIWC/SAV 

Lt  Michael  Perry 
miperrv(a)afiwc.osis.qov 

210-977-2706 
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Guardian 

Guardian  provides  visualization  and  analysis  of  space 
system  and  architecture  susceptibility  to  counter  space 
threats.  Guardian  supports  modeling  of  radio-frequency 
jamming,  air-,  ground-,  and  space-based  laser 
phenomenologies,  high-power  microwave  threats,  and 
direct-ascent  antisatellite  systems.  Guardian  has  the 
capability  to  model  the  interruption  of  system 
commanding,  target  imaging,  and  data  download  to 
ground  stations.  Guardian  has  been  used  to  explore  the 
effects  of  jamming  the  uplink  communications  of 
commercial  satellite  architectures. 

Aerospace  Corp. 

Dr.  James  Gee 

310-336-5871 

Integrated  Modeling 
and  Analysis  Suite 
(IMAS) 

IMAS  models  missile  launches  to  determine  origin  and 
threat.  It  is  used  to  develop  inputs  for  Integrated  Theater 
Warning  and  Attack  Assessment  end-to-end  system 
integrity  tests. 

NORAD/USSPACECO 

M/J6C 

Mr.  Ron  Stephens 

ronald.stephens(S)peter 

son.af.mil 

719-554-9704 

Joint  Integrated 
Contingency  Model 
(JICM) 

JICM  is  a  global  system  developed  to  support  balance 
assessment,  contingency  analysis,  and  military  war 
gaming,  and  it  addresses  both  major  and  smaller 
regional  contingencies.  The  JICM  is  fundamentally  a 
model  of  the  world  in  which  functional  models  of  combat 
and  visualization  tools  are  embedded.  The  JICM 
includes  models  offeree  and  logistics  deployment, 
ground  and  air  combat,  and  ballistic  missiles  and  missile 
defenses.  It  provides  the  user  with  a  graphical  view  of 
the  developing  combat  situation.  Over  the  course  of  a 
JICM  run,  histories  of  outputs  are  retained  that  can  be 
plotted  to  highlight  trends  over  time  of  analyst-selected 
measures  of  outcome.  JICM  modeling  includes  fire 
suppression  by  artillery,  the  effects  of  air  interdiction  on 
ground-force  operations,  combined  arms  effects,  and 
tactical  C3I  effects.  The  effects  of  a  wide  variety  of 
current  and  projected  weapons  are  reflected  in  the  JICM 
design,  including  advanced  antiarmor  systems  (such  as 
sensor-fused  weapons). 

Center  for  Army 

Analysis  (CAA) 

CPT  Max  Moore 

703-806-5593 

moore(S)caa.arrnv.smil. 

mil 

Laser  Threat  Analysis 
System  (LTAS) 

LTAS  is  a  comprehensive  computer  modeling  and 
simulation  environment  for  assessing  the  operational 
impact  of  optical  directed-energy  weapons  and 
countermeasures.  LTAS  encompasses  the  solution 
spectrum  from  physical  process  models  through 
comprehensive  threat  engagement  models. 

AFIWC/SAA 

Jack  Labo 

ialabo(5)afiwc.osis.qov 

210-977-2427 

Model  for  Analysis  of 
Sensor  Coverage 
(MASC) 

A  Windows-based  application  for  computing  the  terrain- 
masked  line-of-sight  coverage  of  ground,  air  and  space- 
based  sensors.  Ground-based  and  airborne  sensor 
coverage  can  be  displayed  in  2  and  3D  while  satellite 
line-of-sight  coverage  is  displayed  as  a  2D  map. 

NORAD 

Ms.  Isabelle  Julien 

isa  belle.  iulien(5)peterso 

n. af.mil 

719-554-3781 
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NORAD / 
USSPACECOM 
Communications 
Simulation  System 
(NUCSS) 

NUCSS  replicates  the  communications  string  of  the 
missile  warning  component  of  Integrated  Tactical 

Warning  and  Attack  Assessment  (ITW/AA).  The  model  is 
maintained  to  reflect  the  current  operational  ITW/AA 
configuration.  It  provides  (1)  a  performance  audit  of  the 
current  ITW/AA  system  under  different  threat  scenarios 
and  stress  events  such  as  link/node  outages  and 
degradation  of  the  communications  links,  (2)  a  method  to 
evaluate  technical  development  of  the  system  and  to 
improve  its  performance,  and  (3)  a  road  map  for 
incorporating  future  mission  capabilities  into  the  ITW/AA 
communications  system.  The  simulation  is  able  to 
federate  under  High  Level  Architecture  with  other 
models. 

NORAD 

Dr.  Roy  Mitchell 

rov.mitchell(5)peterson.a 

f.mil 

719-554-3718 

Space  Battle 

Manager  Core 

System  (SBMCS) 

SBMCS  furnishes  warfighters  with  operationally  relevant 
space  planning  and  execution  information  and  tools  to 
support  their  missions.  SBMCS  is  both  a  client/server- 
based  information  system  accessible  via  Secure  Internet 
Protocol  Router  Network  (SIPRNET)  and  a  deployable, 
non-server-connected,  stand-alone  system.  The 
client/server-based  configuration  of  SBMCS  offers  a 
comprehensive  capability  to  assess  the  space  order  of 
battle,  space  system  resource  status,  satellite  overfly 
and  look-angle  information,  and  navigational  accuracy  to 
anyone  on  the  SIPRNET. 

AFSPC/DOO 

Capt  David  Stone 

719-554-6207  or 
USSPACECOM/J36 

David  Newton 

david.newton(5)peterson 

.af.mil 

719-554-3014 

http://fwwebops.sbmcs. 

usspace.spacecom.smil 

. mil/index. html 

Spacecraft 

Simulation  Toolkit 
(SST) 

The  SST  is  an  advanced,  flexible  development 
environment  for  the  modeling  of  spacecraft  and  their 
environment.  The  SST  is  based  on  state-of-the-art 
simulation  methods  and  accurate  physical 
phenomenology.  It’s  an  object-oriented  system 
consisting  of  software  objects  that  simulate  the  various 
systems  and  subsystems  of  the  physical  spacecraft.  The 
toolkit  provides  the  ability  to  integrate  the  software 
objects  together  into  a  simulation  of  either  a  complete 
spacecraft  system  or  a  spacecraft  subsystem.  A  key 
feature  of  the  SST  is  its  flexibility  to  be  reconfigured  to 
meet  a  wide  variety  of  requirements  in  engineering, 
simulation,  operations  and  training.  Simulations  that 
have  been  or  are  being  developed  include  Ultra 
Lightweight  Imaging  Technology  Experiment,  Space- 
Based  Radar  AMTI/GMTI,  Global  Positioning  System, 
Hyperspectral  Imaging,  Advanced  Geosynchronous 
Studies,  and  the  Next  Generation  Space  Telescope. 

Air  Force  Research 
Laboratory 

Dr.  Rich  de  Jonckheere 

rich.deionckheere(S)vs.a 

frl.af.mil 

505-846-5054 

Strategic  and  Theater 
Operations  Research 
Model  (STORM) 

STORM  will  support  in-depth  analysis  of  the  campaign- 
level  contributions  of  air  and  space  power.  STORM  is  a 
multisided,  object-oriented,  stochastic  computer 
simulation  of  military  operations  across  the  air,  space, 
land,  and  maritime  domains.  The  simulation  is  being 
designed  and  built  expressly  to  examine  issues  involving 
the  utility  and  effectiveness  of  air  and  space  power  in  a 
theater-level,  joint  warfighting  context. 

USAF/AFSAA/SAAP 

Maj  Bryn  Turner 

703-588-8674 

brvn.turner(a)petnaqon.a 

f.mil 
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System  Effectiveness 
Analysis  Simulation 
(SEAS) 

SEAS  is  a  PC-hosted,  many-on-many,  stochastic, 
theater-wide,  multi-mission-level  model.  It  is  typically 
used  for  military  utility  analyses  of  present  and  future 
space  systems  to  explore  combat  outcome  sensitivities 
to  C4ISR  (command,  control,  communication, 
computers,  intelligence,  surveillance,  and 
reconnaissance)  operational  concepts  and  force 
structures.  By  modeling  the  explicit  causal  link  from 
sensor-to-shooter,  SEAS  is  able  to  show  the  emergent 
nonlinear  behavioral  impact  of  C4ISR  on 
spatial/temporal  maneuver  and  attrition  of  terrestrial 
forces.  SEAS  2  is  a  mission  model  in  the  Air  Force 
analysis  toolkit  at  http://www.xo.hq.af.mil/xoc/xoca/afsat 

Capt  Jeremy  Noel 
(USAF/SMC/XRDM) 

310-363-0765 

ieremv.noel(a)losanqele 

s. af.mil 

Sensor  Platform 
Allocation  Tool 
(SPAAT) 

SPAAT  is  an  ISR  force  structure  analysis  tool.  It  is  a 
mixed-integer  program  to  select  sensor  architectures 
based  on  target  coverage  and  cost  constraints.  SPAAT 
is  used  to  determine  the  optimal  mix  of  sensors  and 
platforms  required  to  accomplish  the  reconnaissance 
and  surveillance  mission.  This  optimization  fits  in  the 
overall  picture  of  the  OODA  (observe,  orient,  decide  and 
act)  loop  at  the  orient/decide  phase.  ISR  optimization 
bounds  the  feasible  region  of  the  trade  space.  An  ISR 
mix  that  produces  improved  battlespace  knowledge  can 
be  fed  into  campaign  or  mission  models  to 
illustrate/quantify  the  military  worth  of  ISR. 

AFSAA/SAF 

LtCol  Jeff  Smith 

703-588-8682 

ieff.smith(5)pentaqon.af. 

mil 

Warning 

Warning  is  a  PC-based,  graphical  strategic  ballistic 
missile  warning  analysis  model  designed  to  estimate 
warning  time  available  to  specified  targets  from  launches 
made  from  specified  geographic  areas.  Its  outputs  can 
be  interpreted  as  the  probability  that  missiles  fired  from  a 
particular  area  were  detected  and  reported. 

NORAD 

Ms.  Isabelle  Julien 

isa  belle.  iulien(S)peterso 

n. af.mil 

719-554-3781 
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